Files
2026-10-27 17:41:17 +00:00

365 lines
12 KiB
Markdown

# SpectralForensics
SpectralForensics is a Python library for explainable digital image forensics.
It extracts interpretable descriptors from the frequency, spatial, and
residual-noise domains and provides interactive tools for exploring the
resulting data.
The library contains two complementary extraction paths:
- **RGB extraction** preserves the three color channels, computes
channel-wise descriptors, and aggregates them into image-level values.
- **Grayscale extraction** converts the image to one luminance plane and
reproduces the original SpectralForensics formulation.
The project is being developed to support research into AI-generated image
detection, synthetic-media analysis, diffusion-model fingerprints, and
explainable machine-learning pipelines.
> [!NOTE]
> SpectralForensics is under active development. The public API and extracted
> features may change before the first stable release.
## Features
### Spectral analysis
- Spectral entropy
- Directional anisotropy
- Dominant radial frequency
- Energy density and mean energy
- High-frequency energy ratio
- Normalized FFT spectra for visualization
### Spatial analysis
- Mean brightness and darkness
- Sobel gradient magnitude
- Local entropy
- Local variance
- Normalized visual maps
### Residual-noise analysis
- Residual-noise energy
- Residual-noise standard deviation
- Global residual standard deviation in RGB mode
- Mean absolute residual noise
- Raw and normalized residual-noise maps
### Dataset and visualization tools
- RGB or grayscale extraction from one image or a complete folder
- Sequential renaming of real and AI-generated image datasets
- Optional CSV export
- JPG, JPEG, PNG, BMP, TIFF, and WebP support
- Interactive line, bar, scatter, box, and correlation plots
- Dynamic variable and category filtering
## Requirements
- Python 3.10 or newer
- A graphical desktop environment for interactive plotting
- Tkinter, normally included with standard Python installations
## Installation
```bash
git clone https://gittea.kl3z00.com/kl3z/spectralforensics
cd spectralforensics
python -m pip install -e .
```
The package is not yet advertised as a stable PyPI release.
## Public extraction API
| Function | Input | Color mode | Default size |
| --- | --- | --- | --- |
| `extract_features` | Image or folder | Selected with `color_mode` | RGB: `1024x1024`; grayscale: `255x255` |
| `extract_rgb_features` | Single image | RGB | `1024x1024` |
| `extract_grayscale_features` | Single image | Grayscale | `255x255` |
`extract_features` is the high-level dataset function. RGB remains its default
mode for backward compatibility with the current library.
## Quick start
### Extract RGB features from one image
```python
from spectralforensics import extract_rgb_features
rgb_features = extract_rgb_features("images/sample.jpg",output="quantitative",)
print(rgb_features["color_mode"])
print(rgb_features["spectral_entropy"])
print(rgb_features["mean_gradient"])
print(rgb_features["residual_noise_std_global"])
```
### Extract grayscale features from one image
```python
from spectralforensics import extract_grayscale_features
grayscale_features = extract_grayscale_features("images/sample.jpg",output="quantitative",)
print(grayscale_features["color_mode"])
print(grayscale_features["spectral_entropy"])
print(grayscale_features["mean_gradient"])
print(grayscale_features["residual_noise_std"])
```
Both direct extraction functions return a dictionary containing metadata and
the requested descriptors.
### Process a folder in either mode
```python
from spectralforensics import extract_features
rgb_data = extract_features("images/",color_mode="rgb",output="quantitative",output_csv="rgb_features.csv",)
grayscale_data = extract_features("images/",color_mode="grayscale",output="quantitative",output_csv="grayscale_features.csv",)
```
For a folder, `extract_features` returns a pandas `DataFrame`. Folder traversal
is non-recursive. Set `show_progress=False` to disable the progress bar.
The optional `image_size` argument uses Pillow's `(width, height)` convention.
If omitted, the mode-specific default shown above is used.
### Prepare image filenames
```python
from spectralforensics import rename_generated_images, rename_real_images
real_files = rename_real_images("dataset/real")
generated_files = rename_generated_images("dataset/generated")
```
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and
`gen2.png` patterns. Each function returns the new paths. Renaming is
non-recursive, preserves each image format, and uses temporary unique names to
avoid collisions.
## Extraction modes
The `output` argument controls which values are calculated and returned:
| Mode | Result |
| --- | --- |
| `"all"` | All quantitative descriptors and visual maps |
| `"spectral"` | Spectral descriptors only |
| `"spatial"` | Spatial descriptors only |
| `"residual"` | Residual-noise descriptors only |
| `"quantitative"` | All numerical descriptors without visual maps |
| `"visual"` | Visual maps without quantitative descriptors |
Additional extraction parameters:
| Parameter | Description | Default |
| --- | --- | --- |
| `high_freq_threshold` | Normalized radial threshold for high-frequency energy | `0.65` |
| `center_exclusion` | Half-width of the square excluded around the FFT center | `5` |
| `image_size` | Resized image dimensions as `(width, height)` | Mode-specific |
## RGB and grayscale behavior
### Grayscale path
`features_grayscale.py` converts the input with `Image.convert("L")`. Every
descriptor is therefore computed from one normalized image
`I(x, y) in [0, 1]`.
### RGB path
`features.py` converts the input with `Image.convert("RGB")`, producing
`I(x, y, c)` for `c in {R, G, B}`. The FFT, Sobel gradient, local entropy,
local variance, and median residual are calculated independently for each
channel.
For a channel-wise descriptor `d_c`, the public image-level RGB descriptor is
the arithmetic mean:
```text
d_RGB = (d_R + d_G + d_B) / 3
```
Brightness, darkness, gradient, local entropy, local variance, residual energy,
and mean absolute residual are equivalent to averaging over all pixels and all
three channels. When `R = G = B`, the aggregated RGB descriptors match the
grayscale descriptors up to floating-point precision.
## Extracted metadata and features
### Metadata
| Variable | Description |
| --- | --- |
| `filename` | Name of the processed image file |
| `path` | Original image path |
| `target` | Convenience label inferred from the filename |
| `color_mode` | `"rgb"` or `"grayscale"` |
If a filename contains `gen`, `target` is set to `"0"`. If it contains `real`,
`target` is set to `"1"`. Otherwise it is `None`. This convention is not a
trusted provenance mechanism; research datasets should maintain authoritative
labels separately.
### Spectral descriptors
| Variable | Description |
| --- | --- |
| `spectral_entropy` | Shannon entropy of the normalized Fourier power |
| `anisotropy` | Relative difference between horizontal and vertical central-band energy |
| `dominant_frequency` | Radial FFT-bin distance of the maximum-energy component outside the excluded center |
| `energy_density` | Total spectral energy divided by the number of spatial coefficients |
| `high_frequency_ratio` | Fraction of spectral energy beyond the configured radial threshold |
| `mean_energy` | Mean Fourier power |
In RGB mode, each value is first computed independently for R, G, and B and
then averaged across channels.
### Spatial descriptors
| Variable | Description |
| --- | --- |
| `mean_brightness` | Mean normalized intensity over pixels and available channels |
| `mean_darkness` | Mean intensity complement; equal to `1 - mean_brightness` |
| `mean_gradient` | Mean channel-wise Sobel-gradient magnitude |
| `mean_local_entropy` | Mean local entropy using a disk footprint of radius 3 |
| `mean_local_variance` | Mean local variance using a `7x7` spatial neighborhood |
### Residual-noise descriptors
| Variable | Modes | Description |
| --- | --- | --- |
| `residual_noise_energy` | Both | Mean squared median-filter residual |
| `residual_noise_std` | Both | Grayscale residual standard deviation, or mean of the three channel standard deviations in RGB |
| `residual_noise_std_global` | RGB only | Standard deviation over all RGB residual values jointly |
| `residual_noise_mean_absolute` | Both | Mean absolute residual magnitude |
The median filter uses a `3x3` spatial neighborhood. In RGB mode its channel
size is one, so color channels are never mixed.
## Explainable feature maps
| Variable | Grayscale shape | RGB shape | Description |
| --- | --- | --- | --- |
| `fft_spectrum` | `(H, W)` | `(H, W)` | Normalized aggregate logarithmic spectrum |
| `fft_spectrum_channels` | Not returned | `(H, W, 3)` | Independently normalized R, G, and B logarithmic spectra |
| `brightness_map` | `(H, W)` | `(H, W, 3)` | Normalized image intensities |
| `darkness_map` | `(H, W)` | `(H, W, 3)` | Intensity complements |
| `gradient_map` | `(H, W)` | `(H, W)` | Normalized gradient; RGB channels are averaged before display normalization |
| `local_entropy_map` | `(H, W)` | `(H, W)` | Normalized local entropy; RGB channels are averaged |
| `local_variance_map` | `(H, W)` | `(H, W)` | Normalized local variance; RGB channels are averaged |
| `residual_noise_raw` | `(H, W)` | `(H, W, 3)` | Signed median-filter residual |
| `residual_noise` | `(H, W)` | `(H, W, 3)` | Square-root-enhanced residual visualization |
Visual normalization does not alter the quantitative descriptors.
## Output summary
| Mode | Quantitative descriptors | Visual maps |
| --- | ---: | ---: |
| Grayscale | 14 | 8 |
| RGB | 15 | 9 |
These counts exclude the four metadata fields.
## Interactive visualizations
The plotting functions accept dictionaries, pandas `DataFrame` objects, and
other inputs supported by `pandas.DataFrame`.
```python
from spectralforensics import box_plot, line_plot
data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],}
line_plot(data, x="sample", y="spectral_entropy")
box_plot(data, x="sample", y="anisotropy")
```
![SpectralForensics line plot](img/lines2.jpg)
![SpectralForensics box plot](img/boxplot.jpg)
Available plotting functions:
```python
from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,)
```
### Correlation analysis
```python
from spectralforensics import correlation_plot
correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],}
correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target"],method="pearson",target="target",kind="barh",)
```
![SpectralForensics correlation plot](img/correlation.jpg)
Set `show=False` to return the Matplotlib figure, axes, and correlation tables
without opening the interactive window.
## Interactive chart controls
The chart interface supports:
- X-axis and multiple Y-variable selection
- Category filtering
- Zooming and panning
- Navigation through previous views
- Resetting the current view
- Copying or saving a plot
![SpectralForensics top controls](img/lines.jpg)
![SpectralForensics navigation toolbar](img/menu.jpg)
## Examples
```bash
python examples/line_plot.py
python examples/bar_plot.py
python examples/scatter_plot.py
python examples/box_plot.py
python examples/correlation_plot.py
```
## Project structure
```text
spectralforensics/
|-- examples/ # Plotting examples
|-- img/ # README screenshots
|-- src/
| `-- spectralforensics/
| |-- dataset.py # File and folder processing
| |-- features.py # RGB feature extraction
| |-- features_grayscale.py # Grayscale feature extraction
| |-- plotting*.py # Interactive visualizations
| `-- assets/ # Interface assets
|-- pyproject.toml # Package metadata and dependencies
`-- README.md
```
## Research scope
SpectralForensics is a research and experimentation toolkit. Its descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or generated.
## Authors
- Jose Garcia University of Leiria and Oeste (ULO)
- Patricio Rodrigues University of Leiria and Oeste (ULO)
- Carlos Grilo University of Leiria and Oeste (ULO)
- Rolando Miragaia University of Leiria and Oeste (ULO)
## Contributing
Contribuições?
## Citation
Artigo?