# SpectralForensics SpectralForensics is a Python library for explainable digital image forensics. It extracts interpretable descriptors from the frequency, spatial, and residual-noise domains and provides interactive tools for exploring the resulting data. The library contains two complementary extraction paths: - **RGB extraction** preserves the three color channels, computes channel-wise descriptors, and aggregates them into image-level values. - **Grayscale extraction** converts the image to one luminance plane and reproduces the original SpectralForensics formulation. The project is being developed to support research into AI-generated image detection, synthetic-media analysis, diffusion-model fingerprints, and explainable machine-learning pipelines. > [!NOTE] > SpectralForensics is under active development. The public API and extracted > features may change before the first stable release. ## Features ### Spectral analysis - Spectral entropy - Directional anisotropy - Dominant radial frequency - Energy density and mean energy - High-frequency energy ratio - Normalized FFT spectra for visualization ### Spatial analysis - Mean brightness and darkness - Sobel gradient magnitude - Local entropy - Local variance - Normalized visual maps ### Residual-noise analysis - Residual-noise energy - Residual-noise standard deviation - Global residual standard deviation in RGB mode - Mean absolute residual noise - Raw and normalized residual-noise maps ### Dataset and visualization tools - RGB or grayscale extraction from one image or a complete folder - Sequential renaming of real and AI-generated image datasets - Optional CSV export - JPG, JPEG, PNG, BMP, TIFF, and WebP support - Interactive line, bar, scatter, box, and correlation plots - Dynamic variable and category filtering ## Requirements - Python 3.10 or newer - A graphical desktop environment for interactive plotting - Tkinter, normally included with standard Python installations ## Installation ```bash git clone https://gittea.kl3z00.com/kl3z/spectralforensics cd spectralforensics python -m pip install -e . ``` The package is not yet advertised as a stable PyPI release. ## Public extraction API | Function | Input | Color mode | Default size | | --- | --- | --- | --- | | `extract_features` | Image or folder | Selected with `color_mode` | RGB: `1024x1024`; grayscale: `255x255` | | `extract_rgb_features` | Single image | RGB | `1024x1024` | | `extract_grayscale_features` | Single image | Grayscale | `255x255` | `extract_features` is the high-level dataset function. RGB remains its default mode for backward compatibility with the current library. ## Quick start ### Extract RGB features from one image ```python from spectralforensics import extract_rgb_features rgb_features = extract_rgb_features("images/sample.jpg",output="quantitative",) print(rgb_features["color_mode"]) print(rgb_features["spectral_entropy"]) print(rgb_features["mean_gradient"]) print(rgb_features["residual_noise_std_global"]) ``` ### Extract grayscale features from one image ```python from spectralforensics import extract_grayscale_features grayscale_features = extract_grayscale_features("images/sample.jpg",output="quantitative",) print(grayscale_features["color_mode"]) print(grayscale_features["spectral_entropy"]) print(grayscale_features["mean_gradient"]) print(grayscale_features["residual_noise_std"]) ``` Both direct extraction functions return a dictionary containing metadata and the requested descriptors. ### Process a folder in either mode ```python from spectralforensics import extract_features rgb_data = extract_features("images/",color_mode="rgb",output="quantitative",output_csv="rgb_features.csv",) grayscale_data = extract_features("images/",color_mode="grayscale",output="quantitative",output_csv="grayscale_features.csv",) ``` For a folder, `extract_features` returns a pandas `DataFrame`. Folder traversal is non-recursive. Set `show_progress=False` to disable the progress bar. The optional `image_size` argument uses Pillow's `(width, height)` convention. If omitted, the mode-specific default shown above is used. ### Prepare image filenames ```python from spectralforensics import rename_generated_images, rename_real_images real_files = rename_real_images("dataset/real") generated_files = rename_generated_images("dataset/generated") ``` The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and `gen2.png` patterns. Each function returns the new paths. Renaming is non-recursive, preserves each image format, and uses temporary unique names to avoid collisions. ## Extraction modes The `output` argument controls which values are calculated and returned: | Mode | Result | | --- | --- | | `"all"` | All quantitative descriptors and visual maps | | `"spectral"` | Spectral descriptors only | | `"spatial"` | Spatial descriptors only | | `"residual"` | Residual-noise descriptors only | | `"quantitative"` | All numerical descriptors without visual maps | | `"visual"` | Visual maps without quantitative descriptors | Additional extraction parameters: | Parameter | Description | Default | | --- | --- | --- | | `high_freq_threshold` | Normalized radial threshold for high-frequency energy | `0.65` | | `center_exclusion` | Half-width of the square excluded around the FFT center | `5` | | `image_size` | Resized image dimensions as `(width, height)` | Mode-specific | ## RGB and grayscale behavior ### Grayscale path `features_grayscale.py` converts the input with `Image.convert("L")`. Every descriptor is therefore computed from one normalized image `I(x, y) in [0, 1]`. ### RGB path `features.py` converts the input with `Image.convert("RGB")`, producing `I(x, y, c)` for `c in {R, G, B}`. The FFT, Sobel gradient, local entropy, local variance, and median residual are calculated independently for each channel. For a channel-wise descriptor `d_c`, the public image-level RGB descriptor is the arithmetic mean: ```text d_RGB = (d_R + d_G + d_B) / 3 ``` Brightness, darkness, gradient, local entropy, local variance, residual energy, and mean absolute residual are equivalent to averaging over all pixels and all three channels. When `R = G = B`, the aggregated RGB descriptors match the grayscale descriptors up to floating-point precision. ## Extracted metadata and features ### Metadata | Variable | Description | | --- | --- | | `filename` | Name of the processed image file | | `path` | Original image path | | `target` | Convenience label inferred from the filename | | `color_mode` | `"rgb"` or `"grayscale"` | If a filename contains `gen`, `target` is set to `"0"`. If it contains `real`, `target` is set to `"1"`. Otherwise it is `None`. This convention is not a trusted provenance mechanism; research datasets should maintain authoritative labels separately. ### Spectral descriptors | Variable | Description | | --- | --- | | `spectral_entropy` | Shannon entropy of the normalized Fourier power | | `anisotropy` | Relative difference between horizontal and vertical central-band energy | | `dominant_frequency` | Radial FFT-bin distance of the maximum-energy component outside the excluded center | | `energy_density` | Total spectral energy divided by the number of spatial coefficients | | `high_frequency_ratio` | Fraction of spectral energy beyond the configured radial threshold | | `mean_energy` | Mean Fourier power | In RGB mode, each value is first computed independently for R, G, and B and then averaged across channels. ### Spatial descriptors | Variable | Description | | --- | --- | | `mean_brightness` | Mean normalized intensity over pixels and available channels | | `mean_darkness` | Mean intensity complement; equal to `1 - mean_brightness` | | `mean_gradient` | Mean channel-wise Sobel-gradient magnitude | | `mean_local_entropy` | Mean local entropy using a disk footprint of radius 3 | | `mean_local_variance` | Mean local variance using a `7x7` spatial neighborhood | ### Residual-noise descriptors | Variable | Modes | Description | | --- | --- | --- | | `residual_noise_energy` | Both | Mean squared median-filter residual | | `residual_noise_std` | Both | Grayscale residual standard deviation, or mean of the three channel standard deviations in RGB | | `residual_noise_std_global` | RGB only | Standard deviation over all RGB residual values jointly | | `residual_noise_mean_absolute` | Both | Mean absolute residual magnitude | The median filter uses a `3x3` spatial neighborhood. In RGB mode its channel size is one, so color channels are never mixed. ## Explainable feature maps | Variable | Grayscale shape | RGB shape | Description | | --- | --- | --- | --- | | `fft_spectrum` | `(H, W)` | `(H, W)` | Normalized aggregate logarithmic spectrum | | `fft_spectrum_channels` | Not returned | `(H, W, 3)` | Independently normalized R, G, and B logarithmic spectra | | `brightness_map` | `(H, W)` | `(H, W, 3)` | Normalized image intensities | | `darkness_map` | `(H, W)` | `(H, W, 3)` | Intensity complements | | `gradient_map` | `(H, W)` | `(H, W)` | Normalized gradient; RGB channels are averaged before display normalization | | `local_entropy_map` | `(H, W)` | `(H, W)` | Normalized local entropy; RGB channels are averaged | | `local_variance_map` | `(H, W)` | `(H, W)` | Normalized local variance; RGB channels are averaged | | `residual_noise_raw` | `(H, W)` | `(H, W, 3)` | Signed median-filter residual | | `residual_noise` | `(H, W)` | `(H, W, 3)` | Square-root-enhanced residual visualization | Visual normalization does not alter the quantitative descriptors. ## Output summary | Mode | Quantitative descriptors | Visual maps | | --- | ---: | ---: | | Grayscale | 14 | 8 | | RGB | 15 | 9 | These counts exclude the four metadata fields. ## Interactive visualizations The plotting functions accept dictionaries, pandas `DataFrame` objects, and other inputs supported by `pandas.DataFrame`. ```python from spectralforensics import box_plot, line_plot data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],} line_plot(data, x="sample", y="spectral_entropy") box_plot(data, x="sample", y="anisotropy") ``` ![SpectralForensics line plot](img/lines2.jpg) ![SpectralForensics box plot](img/boxplot.jpg) Available plotting functions: ```python from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,) ``` ### Correlation analysis ```python from spectralforensics import correlation_plot correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],} correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target"],method="pearson",target="target",kind="barh",) ``` ![SpectralForensics correlation plot](img/correlation.jpg) Set `show=False` to return the Matplotlib figure, axes, and correlation tables without opening the interactive window. ## Interactive chart controls The chart interface supports: - X-axis and multiple Y-variable selection - Category filtering - Zooming and panning - Navigation through previous views - Resetting the current view - Copying or saving a plot ![SpectralForensics top controls](img/lines.jpg) ![SpectralForensics navigation toolbar](img/menu.jpg) ## Examples ```bash python examples/line_plot.py python examples/bar_plot.py python examples/scatter_plot.py python examples/box_plot.py python examples/correlation_plot.py ``` ## Project structure ```text spectralforensics/ |-- examples/ # Plotting examples |-- img/ # README screenshots |-- src/ | `-- spectralforensics/ | |-- dataset.py # File and folder processing | |-- features.py # RGB feature extraction | |-- features_grayscale.py # Grayscale feature extraction | |-- plotting*.py # Interactive visualizations | `-- assets/ # Interface assets |-- pyproject.toml # Package metadata and dependencies `-- README.md ``` ## Research scope SpectralForensics is a research and experimentation toolkit. Its descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or generated. ## Authors - Jose Garcia University of Leiria and Oeste (ULO) - Patricio Rodrigues University of Leiria and Oeste (ULO) - Carlos Grilo University of Leiria and Oeste (ULO) - Rolando Miragaia University of Leiria and Oeste (ULO) ## Contributing Contribuições? ## Citation Artigo?