# SpectralForensics SpectralForensics is an open-source Python library for explainable digital image forensics. It extracts interpretable descriptors from the frequency, spatial, and residual-noise domains and provides interactive tools for exploring the resulting data. The project is being developed to support research into AI-generated image detection, synthetic media analysis, diffusion-model fingerprints, and explainable machine-learning pipelines. > [!NOTE] > SpectralForensics is under active development. The public API and extracted features may change before the first stable release. ## Features ### Spectral analysis - Spectral entropy - Anisotropy - Dominant frequency - Energy density and mean energy - High-frequency energy ratio - Normalized FFT spectrum for visualization ### Spatial analysis - Mean brightness and darkness - Gradient magnitude - Local entropy - Local variance - Normalized visual maps for each spatial descriptor ### Residual-noise analysis - Residual-noise energy - Residual-noise standard deviation - Mean absolute residual noise - Raw and normalized residual-noise maps ### Dataset and visualization tools - Feature extraction from a single image or a complete folder - Sequential renaming of real and AI-generated image datasets - Optional CSV export - Support for JPG, JPEG, PNG, BMP, TIFF, and WebP images - Interactive line, bar, scatter, box, and correlation plots - Selection and filtering of multiple variables directly in the plot window ## Requirements - Python 3.10 or newer - A graphical desktop environment for the interactive plotting tools - Tkinter, normally included with standard Python installations ## Installation Clone the repository and install it in editable mode: ```bash git clone https://gittea.kl3z00.com/kl3z/spectralforensics cd spectralforensics python -m pip install -e . ``` The package is not yet advertised as a stable PyPI release. ## Quick start ### Analyse a single image ```python from spectralforensics import extract_features features = extract_features("images/sample.jpg") print(features["spectral_entropy"]) print(features["mean_gradient"]) print(features["residual_noise_energy"]) ``` For a single image, `extract_features` returns a dictionary containing the file metadata and the requested descriptors. ### Analyse a folder ```python from spectralforensics import extract_features data = extract_features("images/",output_csv="spectral_features.csv",output="quantitative",) print(data.head()) ``` When a directory is provided, the function processes supported images in that directory and returns a pandas `DataFrame`. Folder traversal is currently non-recursive. ### Prepare image filenames SpectralForensics can rename all supported images in a folder using the `real` or `gen` prefixes: ```python from spectralforensics import rename_generated_images, rename_real_images real_files = rename_real_images("dataset/real") generated_files = rename_generated_images("dataset/generated") ``` The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and `gen2.png` patterns. Each function returns a list of the new paths. Renaming is non-recursive and preserves each image format. ## Extraction modes Use the `output` argument to control which results are calculated and returned: | Mode | Result | | --- | --- | | `"all"` | All quantitative descriptors and visual maps | | `"spectral"` | Spectral descriptors only | | `"spatial"` | Spatial descriptors only | | `"residual"` | Residual-noise descriptors only | | `"quantitative"` | All numerical descriptors, without visual maps | | `"visual"` | Visual maps, without quantitative descriptors | Example: ```python features = extract_features("images/sample.jpg",image_size=(255, 255),high_freq_threshold=0.65, output="spectral",) ``` # Extracted Features and Metadata The SpectralForensics library extracts a set of quantitative features and interpretable feature maps designed to characterize images in the frequency and spatial domains, including residual noise analysis. ## 1. Metadata | Variable | Description | |----------|-------------| | **`filename`** | Name of the processed image file. | | **`target`** | Image class label (`0 = Real`, `1 = Synthetic`). | --- ## 2. Spectral Features These features characterize the frequency-domain properties of an image using Fourier-based analysis. | Variable | Description | |----------|-------------| | **`spectral_entropy`** | Measures the complexity of the energy distribution in the frequency domain. Higher values indicate a more dispersed spectral energy distribution. | | **`anisotropy`** | Quantifies the difference between horizontal and vertical spectral energy, enabling the identification of directional frequency patterns. | | **`dominant_frequency`** | Measures the distance between the highest-energy frequency component and the center of the Fourier spectrum, representing the dominant spatial frequency of the image. | | **`energy_density`** | Average spectral energy per Fourier transform coefficient. | | **`high_frequency_ratio`** | Proportion of spectral energy concentrated in high-frequency components relative to the total spectral energy. | --- ## 3. Spatial Features These features describe image intensity, local texture, and structural variations in the spatial domain. | Variable | Description | |----------|-------------| | **`mean_brightness`** | Average pixel intensity of the grayscale image. | | **`mean_darkness`** | Average value of the darkness map, indicating the prevalence of dark regions within the image. | | **`mean_gradient`** | Average gradient magnitude, representing the overall presence of edges and intensity transitions. | | **`mean_local_entropy`** | Average local entropy, reflecting the complexity and texture variability of the image. | | **`mean_local_variance`** | Average local intensity variance, characterizing local texture variability. | --- ## 4. Residual Noise Features These features quantify the statistical properties of residual noise obtained through image filtering. | Variable | Description | |----------|-------------| | **`residual_noise_energy`** | Mean energy of the residual noise signal. | | **`residual_noise_std`** | Standard deviation of the residual noise signal. | | **`residual_noise_mean_absolute`** | Mean absolute value of the residual noise signal. | --- ## 5. Explainable Feature Maps In addition to quantitative features, the library generates interpretable maps that enable visual inspection of spatial structures, texture characteristics, frequency distributions, and residual noise patterns. | Variable | Description | |----------|-------------| | **`brightness_map`** | Normalized luminance intensity map (`0 = Black`, `1 = White`). | | **`darkness_map`** | Inverted luminance intensity map (`0 = White`, `1 = Black`). | | **`gradient_map`** | Gradient magnitude map highlighting edges and intensity transitions. | | **`local_entropy_map`** | Local entropy map highlighting regions with increased texture complexity and information variability. | | **`local_variance_map`** | Local variance map highlighting regions with greater local intensity variability. | | **`residual_noise_raw`** | Raw residual noise obtained by subtracting the filtered image from the original image. Preserves the original residual signal and is used for quantitative analysis. | | **`residual_noise_display`** | Visual representation of residual noise after normalization and contrast enhancement. Intended exclusively for qualitative inspection. | | **`fft_spectrum`** | Normalized representation of the Fourier spectrum, illustrating the spatial frequency distribution of the image. | --- ## 6. Feature Summary The following table summarizes the metadata, quantitative features, and explainable maps provided by the library. | Category | Variables | |----------|-----------| | **Metadata** | `filename`, `target` | | **Spectral Features** | `spectral_entropy`, `anisotropy`, `dominant_frequency`, `energy_density`, `high_frequency_ratio` | | **Spatial Features** | `mean_brightness`, `mean_darkness`, `mean_gradient`, `mean_local_entropy`, `mean_local_variance` | | **Residual Noise Features** | `residual_noise_energy`, `residual_noise_std`, `residual_noise_mean_absolute` | | **Explainable Feature Maps** | `brightness_map`, `darkness_map`, `gradient_map`, `local_entropy_map`, `local_variance_map`, `residual_noise_raw`, `residual_noise_display`, `fft_spectrum` | ### Summary of Extracted Information The library provides: - **13 quantitative features**, including spectral, spatial, and residual noise descriptors. - **8 explainable feature maps** for visual inspection and forensic interpretation. - **2 metadata fields** for image identification and class labeling. These outputs support quantitative image characterization, statistical analysis, machine learning workflows, and forensic investigations involving real and synthetically generated images. If a filename contains `gen`, the generated-image target is set to `"0"`. If it contains `real`, the target is set to `"1"`; otherwise, the target is left as `None`. ## Interactive visualizations The plotting functions accept dictionaries, pandas `DataFrame` objects, and other data structures supported by `pandas.DataFrame`. ```python from spectralforensics import line_plot, box_plot data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],} line_plot(data, x="sample", y="spectral_entropy") box_plot(data, x="spectral_entropy", y="anisotropy") ``` ![SpectralForensics line_plot](img/lines2.jpeg) ![SpectralForensics box_plot](img/boxplot.jpeg) Available plotting functions: ```python from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,) ``` ### Correlation analysis ```python from spectralforensics import correlation_plot correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],} correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target", method="pearson", target="target", kind="barh",) ``` ![SpectralForensics box_plot](img/correlation.jpeg) Set `show=False` to generate and retrieve the Matplotlib figure, axes, and correlation tables without opening the interactive window. ## Interactive Chart Controls and Navigation The interactive chart provides a set of controls that allow users to customize data visualization, explore relationships between variables, and interact with the graphical representation. ### 1. Axis Configuration and Variable Selection At the top of the chart, two dropdown menus allow users to select the variables displayed on the **X-axis** and **Y-axis**. - **X-axis:** Supports both categorical (qualitative) and numerical (quantitative) variables, enabling different types of data exploration and comparison. - **Y-axis:** Accepts only numerical (quantitative) variables, ensuring that the selected data can be represented on a continuous numerical scale. When a **categorical variable** is selected for the X-axis, an additional dropdown menu becomes available, allowing users to select specific categories associated with that variable. Two additional buttons are provided: - **Save Image:** Exports the current chart as an image. - **Reset View:** Restores the chart to its initial position and visualization settings. ### 2. Interactive Navigation Toolbar An interactive toolbar is located in the **bottom-left corner of the chart**, providing several tools for navigating, adjusting, and exporting the visualization. The toolbar includes the following controls, described **from right to left**: | Control | Function | |---------|----------| | **Save (Floppy Disk Icon)** | Saves a snapshot of the current chart, allowing users to export the visualization as an image file. | | **Configure Subplots** | Allows users to adjust the chart layout, including subplot dimensions, spacing, and positioning. | | **Zoom (Magnifying Glass Icon)** | Enables users to zoom into a selected region of the chart for more detailed inspection. | | **Pan (Multidirectional Arrows Icon)** | Allows users to move or reposition the chart interactively while maintaining the current zoom level. | | **Forward (Right Arrow Icon)** | Navigates to the next view in the chart's navigation history. | | **Back (Left Arrow Icon)** | Returns to the previous view in the chart's navigation history. | | **Home (House Icon)** | Restores the chart to its original view, resetting the zoom level and axis positioning. | ![SpectralForensics Top bottons](img/lines.jpeg) ![SpectralForensics Bottom bottons](img/menu.jpeg) ### 3. Interactive Data Exploration Together, these controls provide a flexible environment for exploring and interpreting the extracted image features. Users can dynamically select variables, filter categorical data, examine specific regions through zooming and panning, navigate between previous views, and export visualizations for further analysis or documentation. These capabilities support **exploratory data analysis (EDA)**, facilitate the identification of patterns and relationships between extracted features, and improve the interpretability of quantitative image forensic analysis. ## Examples Runnable examples are available in the [`examples`](examples) directory: ```bash python examples/line_plot.py python examples/bar_plot.py python examples/scatter_plot.py python examples/box_plot.py python examples/correlation_plot.py ``` ## Project structure ```text spectralforensics/ |-- examples/ # Usage examples |-- src/ | `-- spectralforensics/ | |-- dataset.py # File and folder processing | |-- features.py # Feature extraction | |-- plotting*.py # Interactive visualization tools | `-- assets/ # Plot interface assets |-- pyproject.toml # Package metadata and dependencies `-- README.md ``` ## Research scope SpectralForensics is intended as a research and experimentation toolkit. Extracted descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or AI-generated. ## Authors - Jose Garcia - Patricio Rodrigues - Carlos Grilo - Rolando Miragaia ## Contributing Issues, suggestions, and contributions are welcome. For substantial changes, open an issue first to discuss the proposed approach and its impact on the public API. ## Citation If you use SpectralForensics in academic work, please cite the repository. A formal citation and publication reference will be added as the project matures.