Efficient Face-Swap-Verification Using PRNU

Facial recognition is becoming the go-to method of identifying users for convenience applications. While great advances have occurred in achieving strong false acceptance and false rejection rates on authentic images, these systems can be vulnerable to face-swap-attacks. This research addresses face-swap-attacks via camera forensics. Whenever an image is modified, there is necessarily an impact to the noise profile (in this case Photo Response Non-Uniformity). Hence, a framework is proposed to enroll the facial recognition camera's noiseprint and assess authenticity on future images based on deviation from expected value. This is done using down-sampling compression to improve run time, where images are further segmented into sub-zones to retain local sensitivity. Framework performance is evaluated by recording identical facial-images using multiple cameras of the same make. Next, a subset is modified via hand-crafted and AI-tool face-swaps. 100% of images are correctly identified as authentic or tampering when using full-image analysis at full-scale. Efficiency is then optimized by dividing the image into sub-zones and applying compression. Run-time is improved to 4.6 msec on CPU, a 99.1% reduction, by applying quarter-scale down-sampling with 16 sub-zones (this retains 93.5% verification accuracy). These results are validated against three existing state-of-the-art algorithms, which in comparison show over-fitting when compressed. This demonstrates that compressed PRNU can be used to efficiently verify facial-images, including against AI facial manipulation tools.

Automated summary

This research proposes a framework to address face-swap-attacks through camera forensics. The framework records the noiseprint of facial recognition cameras and assesses the authenticity of future images based on deviation from expected values. The framework achieves satisfactory results in terms of performance and efficiency by using down-sampling compression and image segmentation.