Crazzy Bugs

CII shield Dashboard

Repository : Link

Overview

Title: CII SHIELD: Offline Parallel AV Pipeline for CII Entities

Problem Statement ID: 1685

Objective: The goal of the project is to create an automated offline file-scanning system using multiple antivirus (AV) engines running in parallel. The system ensures files are scanned for malware, segregates infected files, and generates reports while maintaining Critical Information Infrastructure (CII) compliance.

Problem Statement

Background

  • No single AV solution is perfect. Each antivirus has unique strengths and weaknesses.
  • CII Requirements: Offline, on-premise solution with regular AV updates.

Description

  • Develop a pipeline to scan files using multiple AV engines simultaneously.
  • Files are automatically categorized as Clean or Infected.
  • Clean files remain in the original folder, while infected files are moved to a separate Infected Folder.
  • Generate analysis reports for infected files, including their file paths.
  • Provide a user-friendly dashboard to monitor the scanning process and view results.

Workflow

  1. Target Folder: Files to be scanned reside in a monitored folder.
  2. Parallel AV Scanning: Multiple AV engines scan the files simultaneously.
  3. Infected Folder: Infected files and analysis reports are moved to a dedicated folder.
  4. Dashboard: Real-time updates of scan results, showing:
    • File status: Clean or Infected
    • AV engine reports
  5. Notification: Pop-up or message indicates when scanning is complete.

Flow

Goals

  • Automate the scanning process for files in the Target Folder.
  • Parallel Processing to reduce scanning time.
  • Organize Output: Segregate infected files and log analysis results.
  • Dashboard Interface: Real-time view of scan results.
  • Offline Operation: Ensure no internet dependency for AV updates.

Features

  • Reliability: Accurate identification of infected files with minimal false positives.
  • User-Friendly Dashboard: Intuitive interface displaying scanning results and infection details.
  • Efficiency: Parallel processing for faster file scanning.
  • Antivirus-as-a-Microservice: Each AV runs in an isolated container.
  • AI-Powered Pre-Scanning: File risk analysis for optimized resource usage.
  • Sandbox Threat Analysis: Detailed inspection of quarantined files.
  • Real-Time Dashboard: Results displayed using in-memory database (Redis).

Flow of Operation

  1. Monitor Target Folder: Detects new files for scanning.
  2. Trigger Parallel Scanning: Files are scanned simultaneously by multiple AV engines.
  3. File Categorization:
    • Clean Files: Retained in the Target Folder.
    • Infected Files: Moved to Infected Folder.
  4. Analysis Reports: Generated for each infected file.
  5. Dashboard Update: Real-time display of scan results.
  6. Completion Notification: Message displayed upon scan completion.

Modules

1. File Processing Module

  • Purpose: Monitor the Target Folder and initiate AV scans.
  • Technologies: C++ (File System Watcher, Thread Pool).

2. Parallel AV Scanning Engine

  • Purpose: Execute antivirus scans in parallel.
  • Technologies: Docker, Ansible, Go (for container management).
  • Implementation:
    • Each AV runs in an isolated Docker container.
    • Containers communicate results via NFS/SMB.

3. Result Management & Segregation

  • Purpose: Organize clean and infected files, generate analysis reports.
  • Technologies:
    • C++: File handling.
    • Python: Report generation.
    • Logging: spdlog (C++) or Python's logging module.
  • Output:
    • Clean files remain in the Target Folder.
    • Infected files moved to Infected Folder with:
      • Analysis reports (JSON/XML).
      • File paths logged in a document.

4. Dashboard Interface

  • Purpose: User-friendly web interface to display scan results.
  • Technologies:
    • Frontend: Next.js.
    • Backend: Node.js.
    • Database: SQLite or MongoDB.
  • Features:
    • Real-time scanning progress.
    • Detailed infection reports.
    • AV engine-specific results.

Technologies Used

  • C++/C: File monitoring, queue management, file handling.
  • Python: Report generation, logging.
  • Go: Container management.
  • Node.js: Backend API.
  • Next.js: Frontend dashboard.
  • Docker: Containerization of AV engines.
  • Bash Shell Scripts: Automation for AV configuration.

Implementation Plan

Offline Executable Version

  • Target: High-security CII environments.
  • Components:
    1. Setup:
      • MSI/Executable installer for offline deployment.
      • Installs Docker, AVs, and required dependencies.
    2. File System Monitoring:
      • Detects new files, queues them for scanning.
    3. Virtualization Layer:
      • Runs AVs in Docker containers or VMs.
    4. Result Storage:
      • Local SQLite database for logs and reports.
    5. User Interface:
      • Local web-based dashboard for results.

Cloud/Server-Based Version (API/SDK)

  • Target: Enterprise systems for real-time scanning.
  • Components:
    1. API/SDK Layer:
      • REST/gRPC APIs for submitting files and retrieving results.
    2. File Queue Management:
      • Handles file submissions and triggers AV scans.
    3. Virtualization Layer:
      • Lightweight Docker containers managed by Kubernetes.
    4. Real-Time Dashboard:
      • Hosted dashboard displaying real-time results.

Challenges & Solutions

1. Offline Updates

  • Challenge: Limited frequency of virus definition updates.
  • Solution: Use centralized updates transferred via physical media (e.g., USB drive).

2. Compatibility Issues

  • Challenge: AV engines may conflict in a shared environment.
  • Solution: Use Docker containers for isolation.

3. Resource Management

  • Challenge: High CPU and memory usage.
  • Solution: Use lightweight virtualization.

4. Complex Setup

  • Challenge: Difficulty in maintaining the system.
  • Solution: Provide automated setup scripts.

Final Product

Offline Version

  • Format: MSI/Executable with:
    • Lightweight Docker containers or VM snapshot for AV engines.
    • Local electron desktop app dashboard for result display and analytics.

Cloud/Server Version

  • Format: API-based solution with:
    • Docker containers for AV engines.
    • Hosted dashboard for remote monitoring.