365 lines
12 KiB
Markdown
365 lines
12 KiB
Markdown
# SpectralForensics
|
|
|
|
SpectralForensics is a Python library for explainable digital image forensics.
|
|
It extracts interpretable descriptors from the frequency, spatial, and
|
|
residual-noise domains and provides interactive tools for exploring the
|
|
resulting data.
|
|
|
|
The library contains two complementary extraction paths:
|
|
|
|
- **RGB extraction** preserves the three color channels, computes
|
|
channel-wise descriptors, and aggregates them into image-level values.
|
|
- **Grayscale extraction** converts the image to one luminance plane and
|
|
reproduces the original SpectralForensics formulation.
|
|
|
|
The project is being developed to support research into AI-generated image
|
|
detection, synthetic-media analysis, diffusion-model fingerprints, and
|
|
explainable machine-learning pipelines.
|
|
|
|
> [!NOTE]
|
|
> SpectralForensics is under active development. The public API and extracted
|
|
> features may change before the first stable release.
|
|
|
|
## Features
|
|
|
|
### Spectral analysis
|
|
|
|
- Spectral entropy
|
|
- Directional anisotropy
|
|
- Dominant radial frequency
|
|
- Energy density and mean energy
|
|
- High-frequency energy ratio
|
|
- Normalized FFT spectra for visualization
|
|
|
|
### Spatial analysis
|
|
|
|
- Mean brightness and darkness
|
|
- Sobel gradient magnitude
|
|
- Local entropy
|
|
- Local variance
|
|
- Normalized visual maps
|
|
|
|
### Residual-noise analysis
|
|
|
|
- Residual-noise energy
|
|
- Residual-noise standard deviation
|
|
- Global residual standard deviation in RGB mode
|
|
- Mean absolute residual noise
|
|
- Raw and normalized residual-noise maps
|
|
|
|
### Dataset and visualization tools
|
|
|
|
- RGB or grayscale extraction from one image or a complete folder
|
|
- Sequential renaming of real and AI-generated image datasets
|
|
- Optional CSV export
|
|
- JPG, JPEG, PNG, BMP, TIFF, and WebP support
|
|
- Interactive line, bar, scatter, box, and correlation plots
|
|
- Dynamic variable and category filtering
|
|
|
|
## Requirements
|
|
|
|
- Python 3.10 or newer
|
|
- A graphical desktop environment for interactive plotting
|
|
- Tkinter, normally included with standard Python installations
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
git clone https://gittea.kl3z00.com/kl3z/spectralforensics
|
|
cd spectralforensics
|
|
python -m pip install -e .
|
|
```
|
|
|
|
The package is not yet advertised as a stable PyPI release.
|
|
|
|
## Public extraction API
|
|
|
|
| Function | Input | Color mode | Default size |
|
|
| --- | --- | --- | --- |
|
|
| `extract_features` | Image or folder | Selected with `color_mode` | RGB: `1024x1024`; grayscale: `255x255` |
|
|
| `extract_rgb_features` | Single image | RGB | `1024x1024` |
|
|
| `extract_grayscale_features` | Single image | Grayscale | `255x255` |
|
|
|
|
`extract_features` is the high-level dataset function. RGB remains its default
|
|
mode for backward compatibility with the current library.
|
|
|
|
## Quick start
|
|
|
|
### Extract RGB features from one image
|
|
|
|
```python
|
|
from spectralforensics import extract_rgb_features
|
|
|
|
rgb_features = extract_rgb_features("images/sample.jpg",output="quantitative",)
|
|
print(rgb_features["color_mode"])
|
|
print(rgb_features["spectral_entropy"])
|
|
print(rgb_features["mean_gradient"])
|
|
print(rgb_features["residual_noise_std_global"])
|
|
```
|
|
|
|
### Extract grayscale features from one image
|
|
|
|
```python
|
|
from spectralforensics import extract_grayscale_features
|
|
|
|
grayscale_features = extract_grayscale_features("images/sample.jpg",output="quantitative",)
|
|
print(grayscale_features["color_mode"])
|
|
print(grayscale_features["spectral_entropy"])
|
|
print(grayscale_features["mean_gradient"])
|
|
print(grayscale_features["residual_noise_std"])
|
|
```
|
|
|
|
Both direct extraction functions return a dictionary containing metadata and
|
|
the requested descriptors.
|
|
|
|
### Process a folder in either mode
|
|
|
|
```python
|
|
from spectralforensics import extract_features
|
|
|
|
rgb_data = extract_features("images/",color_mode="rgb",output="quantitative",output_csv="rgb_features.csv",)
|
|
grayscale_data = extract_features("images/",color_mode="grayscale",output="quantitative",output_csv="grayscale_features.csv",)
|
|
```
|
|
|
|
For a folder, `extract_features` returns a pandas `DataFrame`. Folder traversal
|
|
is non-recursive. Set `show_progress=False` to disable the progress bar.
|
|
|
|
The optional `image_size` argument uses Pillow's `(width, height)` convention.
|
|
If omitted, the mode-specific default shown above is used.
|
|
|
|
### Prepare image filenames
|
|
|
|
```python
|
|
from spectralforensics import rename_generated_images, rename_real_images
|
|
|
|
real_files = rename_real_images("dataset/real")
|
|
generated_files = rename_generated_images("dataset/generated")
|
|
```
|
|
|
|
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and
|
|
`gen2.png` patterns. Each function returns the new paths. Renaming is
|
|
non-recursive, preserves each image format, and uses temporary unique names to
|
|
avoid collisions.
|
|
|
|
## Extraction modes
|
|
|
|
The `output` argument controls which values are calculated and returned:
|
|
|
|
| Mode | Result |
|
|
| --- | --- |
|
|
| `"all"` | All quantitative descriptors and visual maps |
|
|
| `"spectral"` | Spectral descriptors only |
|
|
| `"spatial"` | Spatial descriptors only |
|
|
| `"residual"` | Residual-noise descriptors only |
|
|
| `"quantitative"` | All numerical descriptors without visual maps |
|
|
| `"visual"` | Visual maps without quantitative descriptors |
|
|
|
|
Additional extraction parameters:
|
|
|
|
| Parameter | Description | Default |
|
|
| --- | --- | --- |
|
|
| `high_freq_threshold` | Normalized radial threshold for high-frequency energy | `0.65` |
|
|
| `center_exclusion` | Half-width of the square excluded around the FFT center | `5` |
|
|
| `image_size` | Resized image dimensions as `(width, height)` | Mode-specific |
|
|
|
|
## RGB and grayscale behavior
|
|
|
|
### Grayscale path
|
|
|
|
`features_grayscale.py` converts the input with `Image.convert("L")`. Every
|
|
descriptor is therefore computed from one normalized image
|
|
`I(x, y) in [0, 1]`.
|
|
|
|
### RGB path
|
|
|
|
`features.py` converts the input with `Image.convert("RGB")`, producing
|
|
`I(x, y, c)` for `c in {R, G, B}`. The FFT, Sobel gradient, local entropy,
|
|
local variance, and median residual are calculated independently for each
|
|
channel.
|
|
|
|
For a channel-wise descriptor `d_c`, the public image-level RGB descriptor is
|
|
the arithmetic mean:
|
|
|
|
```text
|
|
d_RGB = (d_R + d_G + d_B) / 3
|
|
```
|
|
|
|
Brightness, darkness, gradient, local entropy, local variance, residual energy,
|
|
and mean absolute residual are equivalent to averaging over all pixels and all
|
|
three channels. When `R = G = B`, the aggregated RGB descriptors match the
|
|
grayscale descriptors up to floating-point precision.
|
|
|
|
## Extracted metadata and features
|
|
|
|
### Metadata
|
|
|
|
| Variable | Description |
|
|
| --- | --- |
|
|
| `filename` | Name of the processed image file |
|
|
| `path` | Original image path |
|
|
| `target` | Convenience label inferred from the filename |
|
|
| `color_mode` | `"rgb"` or `"grayscale"` |
|
|
|
|
If a filename contains `gen`, `target` is set to `"0"`. If it contains `real`,
|
|
`target` is set to `"1"`. Otherwise it is `None`. This convention is not a
|
|
trusted provenance mechanism; research datasets should maintain authoritative
|
|
labels separately.
|
|
|
|
### Spectral descriptors
|
|
|
|
| Variable | Description |
|
|
| --- | --- |
|
|
| `spectral_entropy` | Shannon entropy of the normalized Fourier power |
|
|
| `anisotropy` | Relative difference between horizontal and vertical central-band energy |
|
|
| `dominant_frequency` | Radial FFT-bin distance of the maximum-energy component outside the excluded center |
|
|
| `energy_density` | Total spectral energy divided by the number of spatial coefficients |
|
|
| `high_frequency_ratio` | Fraction of spectral energy beyond the configured radial threshold |
|
|
| `mean_energy` | Mean Fourier power |
|
|
|
|
In RGB mode, each value is first computed independently for R, G, and B and
|
|
then averaged across channels.
|
|
|
|
### Spatial descriptors
|
|
|
|
| Variable | Description |
|
|
| --- | --- |
|
|
| `mean_brightness` | Mean normalized intensity over pixels and available channels |
|
|
| `mean_darkness` | Mean intensity complement; equal to `1 - mean_brightness` |
|
|
| `mean_gradient` | Mean channel-wise Sobel-gradient magnitude |
|
|
| `mean_local_entropy` | Mean local entropy using a disk footprint of radius 3 |
|
|
| `mean_local_variance` | Mean local variance using a `7x7` spatial neighborhood |
|
|
|
|
### Residual-noise descriptors
|
|
|
|
| Variable | Modes | Description |
|
|
| --- | --- | --- |
|
|
| `residual_noise_energy` | Both | Mean squared median-filter residual |
|
|
| `residual_noise_std` | Both | Grayscale residual standard deviation, or mean of the three channel standard deviations in RGB |
|
|
| `residual_noise_std_global` | RGB only | Standard deviation over all RGB residual values jointly |
|
|
| `residual_noise_mean_absolute` | Both | Mean absolute residual magnitude |
|
|
|
|
The median filter uses a `3x3` spatial neighborhood. In RGB mode its channel
|
|
size is one, so color channels are never mixed.
|
|
|
|
## Explainable feature maps
|
|
|
|
| Variable | Grayscale shape | RGB shape | Description |
|
|
| --- | --- | --- | --- |
|
|
| `fft_spectrum` | `(H, W)` | `(H, W)` | Normalized aggregate logarithmic spectrum |
|
|
| `fft_spectrum_channels` | Not returned | `(H, W, 3)` | Independently normalized R, G, and B logarithmic spectra |
|
|
| `brightness_map` | `(H, W)` | `(H, W, 3)` | Normalized image intensities |
|
|
| `darkness_map` | `(H, W)` | `(H, W, 3)` | Intensity complements |
|
|
| `gradient_map` | `(H, W)` | `(H, W)` | Normalized gradient; RGB channels are averaged before display normalization |
|
|
| `local_entropy_map` | `(H, W)` | `(H, W)` | Normalized local entropy; RGB channels are averaged |
|
|
| `local_variance_map` | `(H, W)` | `(H, W)` | Normalized local variance; RGB channels are averaged |
|
|
| `residual_noise_raw` | `(H, W)` | `(H, W, 3)` | Signed median-filter residual |
|
|
| `residual_noise` | `(H, W)` | `(H, W, 3)` | Square-root-enhanced residual visualization |
|
|
|
|
Visual normalization does not alter the quantitative descriptors.
|
|
|
|
## Output summary
|
|
|
|
| Mode | Quantitative descriptors | Visual maps |
|
|
| --- | ---: | ---: |
|
|
| Grayscale | 14 | 8 |
|
|
| RGB | 15 | 9 |
|
|
|
|
These counts exclude the four metadata fields.
|
|
|
|
## Interactive visualizations
|
|
|
|
The plotting functions accept dictionaries, pandas `DataFrame` objects, and
|
|
other inputs supported by `pandas.DataFrame`.
|
|
|
|
```python
|
|
from spectralforensics import box_plot, line_plot
|
|
|
|
data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],}
|
|
|
|
line_plot(data, x="sample", y="spectral_entropy")
|
|
box_plot(data, x="sample", y="anisotropy")
|
|
```
|
|
|
|

|
|

|
|
|
|
Available plotting functions:
|
|
|
|
```python
|
|
from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,)
|
|
```
|
|
|
|
### Correlation analysis
|
|
|
|
```python
|
|
from spectralforensics import correlation_plot
|
|
correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],}
|
|
correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target"],method="pearson",target="target",kind="barh",)
|
|
```
|
|
|
|

|
|
|
|
Set `show=False` to return the Matplotlib figure, axes, and correlation tables
|
|
without opening the interactive window.
|
|
|
|
## Interactive chart controls
|
|
|
|
The chart interface supports:
|
|
|
|
- X-axis and multiple Y-variable selection
|
|
- Category filtering
|
|
- Zooming and panning
|
|
- Navigation through previous views
|
|
- Resetting the current view
|
|
- Copying or saving a plot
|
|
|
|

|
|
|
|

|
|
|
|
## Examples
|
|
|
|
```bash
|
|
python examples/line_plot.py
|
|
python examples/bar_plot.py
|
|
python examples/scatter_plot.py
|
|
python examples/box_plot.py
|
|
python examples/correlation_plot.py
|
|
```
|
|
|
|
## Project structure
|
|
|
|
```text
|
|
spectralforensics/
|
|
|-- examples/ # Plotting examples
|
|
|-- img/ # README screenshots
|
|
|-- src/
|
|
| `-- spectralforensics/
|
|
| |-- dataset.py # File and folder processing
|
|
| |-- features.py # RGB feature extraction
|
|
| |-- features_grayscale.py # Grayscale feature extraction
|
|
| |-- plotting*.py # Interactive visualizations
|
|
| `-- assets/ # Interface assets
|
|
|-- pyproject.toml # Package metadata and dependencies
|
|
`-- README.md
|
|
```
|
|
|
|
## Research scope
|
|
|
|
SpectralForensics is a research and experimentation toolkit. Its descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or generated.
|
|
|
|
## Authors
|
|
|
|
- Jose Garcia University of Leiria and Oeste (ULO)
|
|
- Patricio Rodrigues University of Leiria and Oeste (ULO)
|
|
- Carlos Grilo University of Leiria and Oeste (ULO)
|
|
- Rolando Miragaia University of Leiria and Oeste (ULO)
|
|
|
|
## Contributing
|
|
|
|
Contribuições?
|
|
|
|
## Citation
|
|
|
|
Artigo?
|