This repository contains the code and resources for the final year thesis project titled "Development of DNN based Deepfake Video Detection system using multi-modality features," submitted in partial fulfillment of the requirements for the Bachelor of Sciences in Computer and Information Sciences at the Pakistan Institute of Engineering and Applied Sciences (PIEAS).

The project, aims to detect deepfake videos by leveraging both optical artifacts and Transdermal Optical Imaging (TOI) features through a deep learning approach.

We utilized and reproduced concepts from the following research papers:

• Deepfake Video Detection Using Convolutional Vision Transformer
• TALL: Thumbnail Layout for Deepfake Video Detection

• Project Description
• Features
• Technologies Used
• Installation
• Technical Details
• Acknowledgements
• Contact

Deepfake technology poses a significant threat to media credibility and cybersecurity by creating highly realistic falsified videos that can:

• Spread misinformation
• Damage reputations
• Facilitate crimes such as fraud and propaganda

This project addresses the challenge of detecting deepfakes by combining:

• Optical Artifacts (e.g., facial landmark distortions, unnatural movements)
• TOI Features (e.g., subtle blood flow patterns beneath the skin)

A Deep Neural Network (DNN) approach is used to analyze these multi-modality features to provide a robust and accurate detection system.

A web application complements the backend, offering real-time video authenticity verification.

• Multi-Modality Feature Extraction: Combines optical and TOI features.
• Thumbnail Layout (TALL): Efficient video representation using compact thumbnails.
• Deep Learning Models:
  • CViT (Convolutional Vision Transformer)
  • TALL-TimeSformer (Transformer with thumbnail input)
• Web Application: Real-time video analysis via browser-based interface.

• Programming Language: Python 3.10
• Frameworks:
  • PyTorch (Deep Learning)
  • Next.js (Frontend)
• Database & Hosting:
  • Supabase (Storage/DB)
  • Modal (Model Hosting)
  • Vercel (Frontend Deployment)
• Other Packages: MTCNN, NumPy, Transformers, OpenCV, WandB, Facenet-PyTorch, Deepspeed

git clone https://github.com/muhammadhamzaazhar/DeepFake-Video-Detection.git cd DeepFake-Video-Detection

• Make sure Python 3.10 is installed on your system.
• Then, install the required Python packages by running:

pip install -r requirements.txt

To make a prediction on a video:

1. Download the pretrained model and config.js from Hugging Face.
2. Place the downloaded files inside the weights/ directory.
3. Run the prediction script using:

python predict.py "path/to/your/video.mp4"

For complete details on:

• Dataset sources and structure
• Preprocessing pipeline (face detection, skin segmentation, TOI heatmaps)
• Model architecture and training methodology

please refer to the Thesis Report.

• Supervisor: Dr. Asifullah Khan, DCIS PIEAS
• Co-Supervisor: Dr. Abdul Majid, DCIS PIEAS
• Research Labs:
  • PRLab
  • PIEAS AI Center (PAIC)
  • Center for Mathematical Sciences (CMS)

For inquiries or feedback, please reach out to:

• Email: hamzaazharmuhammad@gmail.com
• LinkedIn: https://www.linkedin.com/in/muhammad-hamza-azhar

• Email: haiderali10986@gmail.com
• LinkedIn: https://www.linkedin.com/in/haider-ali-aurangzaib