346 lines
14 KiB
Markdown
346 lines
14 KiB
Markdown
# SpectralForensics
|
|
|
|
SpectralForensics is an open-source Python library for explainable digital image forensics. It extracts interpretable descriptors from the frequency, spatial, and residual-noise domains and provides interactive tools for exploring the resulting data.
|
|
|
|
The project is being developed to support research into AI-generated image detection, synthetic media analysis, diffusion-model fingerprints, and explainable machine-learning pipelines.
|
|
|
|
> [!NOTE]
|
|
> SpectralForensics is under active development. The public API and extracted features may change before the first stable release.
|
|
|
|
## Features
|
|
|
|
### Spectral analysis
|
|
|
|
- Spectral entropy
|
|
- Anisotropy
|
|
- Dominant frequency
|
|
- Energy density and mean energy
|
|
- High-frequency energy ratio
|
|
- Normalized FFT spectrum for visualization
|
|
|
|
### Spatial analysis
|
|
|
|
- Mean brightness and darkness
|
|
- Gradient magnitude
|
|
- Local entropy
|
|
- Local variance
|
|
- Normalized visual maps for each spatial descriptor
|
|
|
|
### Residual-noise analysis
|
|
|
|
- Residual-noise energy
|
|
- Residual-noise standard deviation
|
|
- Mean absolute residual noise
|
|
- Raw and normalized residual-noise maps
|
|
|
|
### Dataset and visualization tools
|
|
|
|
- Feature extraction from a single image or a complete folder
|
|
- Sequential renaming of real and AI-generated image datasets
|
|
- Optional CSV export
|
|
- Support for JPG, JPEG, PNG, BMP, TIFF, and WebP images
|
|
- Interactive line, bar, scatter, box, and correlation plots
|
|
- Selection and filtering of multiple variables directly in the plot window
|
|
|
|
## Requirements
|
|
|
|
- Python 3.10 or newer
|
|
- A graphical desktop environment for the interactive plotting tools
|
|
- Tkinter, normally included with standard Python installations
|
|
|
|
## Installation
|
|
|
|
Clone the repository and install it in editable mode:
|
|
|
|
```bash
|
|
git clone https://gittea.kl3z00.com/kl3z/spectralforensics
|
|
cd spectralforensics
|
|
python -m pip install -e .
|
|
```
|
|
|
|
The package is not yet advertised as a stable PyPI release.
|
|
|
|
## Quick start
|
|
|
|
### Analyse a single image
|
|
|
|
```python
|
|
from spectralforensics import extract_features
|
|
|
|
features = extract_features("images/sample.jpg")
|
|
|
|
print(features["spectral_entropy"])
|
|
print(features["mean_gradient"])
|
|
print(features["residual_noise_energy"])
|
|
```
|
|
|
|
For a single image, `extract_features` returns a dictionary containing the file metadata and the requested descriptors.
|
|
|
|
### Analyse a folder
|
|
|
|
```python
|
|
from spectralforensics import extract_features
|
|
|
|
data = extract_features("images/",output_csv="spectral_features.csv",output="quantitative",)
|
|
|
|
print(data.head())
|
|
```
|
|
|
|
When a directory is provided, the function processes supported images in that directory and returns a pandas `DataFrame`. Folder traversal is currently non-recursive.
|
|
|
|
### Prepare image filenames
|
|
|
|
SpectralForensics can rename all supported images in a folder using the
|
|
`real` or `gen` prefixes:
|
|
|
|
```python
|
|
from spectralforensics import rename_generated_images, rename_real_images
|
|
|
|
real_files = rename_real_images("dataset/real")
|
|
generated_files = rename_generated_images("dataset/generated")
|
|
```
|
|
|
|
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`,
|
|
and `gen2.png` patterns. Each function returns a list of the new paths.
|
|
Renaming is non-recursive and preserves each image format.
|
|
|
|
## Extraction modes
|
|
|
|
Use the `output` argument to control which results are calculated and returned:
|
|
|
|
| Mode | Result |
|
|
| --- | --- |
|
|
| `"all"` | All quantitative descriptors and visual maps |
|
|
| `"spectral"` | Spectral descriptors only |
|
|
| `"spatial"` | Spatial descriptors only |
|
|
| `"residual"` | Residual-noise descriptors only |
|
|
| `"quantitative"` | All numerical descriptors, without visual maps |
|
|
| `"visual"` | Visual maps, without quantitative descriptors |
|
|
|
|
Example:
|
|
|
|
```python
|
|
features = extract_features("images/sample.jpg",image_size=(255, 255),high_freq_threshold=0.65, output="spectral",)
|
|
```
|
|
|
|
# Extracted Features and Metadata
|
|
|
|
The SpectralForensics library extracts a set of quantitative features and interpretable feature maps designed to characterize images in the frequency and spatial domains, including residual noise analysis.
|
|
|
|
## 1. Metadata
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| **`filename`** | Name of the processed image file. |
|
|
| **`target`** | Image class label (`0 = Real`, `1 = Synthetic`). |
|
|
|
|
---
|
|
|
|
## 2. Spectral Features
|
|
|
|
These features characterize the frequency-domain properties of an image using Fourier-based analysis.
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| **`spectral_entropy`** | Measures the complexity of the energy distribution in the frequency domain. Higher values indicate a more dispersed spectral energy distribution. |
|
|
| **`anisotropy`** | Quantifies the difference between horizontal and vertical spectral energy, enabling the identification of directional frequency patterns. |
|
|
| **`dominant_frequency`** | Measures the distance between the highest-energy frequency component and the center of the Fourier spectrum, representing the dominant spatial frequency of the image. |
|
|
| **`energy_density`** | Average spectral energy per Fourier transform coefficient. |
|
|
| **`high_frequency_ratio`** | Proportion of spectral energy concentrated in high-frequency components relative to the total spectral energy. |
|
|
|
|
---
|
|
|
|
## 3. Spatial Features
|
|
|
|
These features describe image intensity, local texture, and structural variations in the spatial domain.
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| **`mean_brightness`** | Average pixel intensity of the grayscale image. |
|
|
| **`mean_darkness`** | Average value of the darkness map, indicating the prevalence of dark regions within the image. |
|
|
| **`mean_gradient`** | Average gradient magnitude, representing the overall presence of edges and intensity transitions. |
|
|
| **`mean_local_entropy`** | Average local entropy, reflecting the complexity and texture variability of the image. |
|
|
| **`mean_local_variance`** | Average local intensity variance, characterizing local texture variability. |
|
|
|
|
---
|
|
|
|
## 4. Residual Noise Features
|
|
|
|
These features quantify the statistical properties of residual noise obtained through image filtering.
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| **`residual_noise_energy`** | Mean energy of the residual noise signal. |
|
|
| **`residual_noise_std`** | Standard deviation of the residual noise signal. |
|
|
| **`residual_noise_mean_absolute`** | Mean absolute value of the residual noise signal. |
|
|
|
|
---
|
|
|
|
## 5. Explainable Feature Maps
|
|
|
|
In addition to quantitative features, the library generates interpretable maps that enable visual inspection of spatial structures, texture characteristics, frequency distributions, and residual noise patterns.
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| **`brightness_map`** | Normalized luminance intensity map (`0 = Black`, `1 = White`). |
|
|
| **`darkness_map`** | Inverted luminance intensity map (`0 = White`, `1 = Black`). |
|
|
| **`gradient_map`** | Gradient magnitude map highlighting edges and intensity transitions. |
|
|
| **`local_entropy_map`** | Local entropy map highlighting regions with increased texture complexity and information variability. |
|
|
| **`local_variance_map`** | Local variance map highlighting regions with greater local intensity variability. |
|
|
| **`residual_noise_raw`** | Raw residual noise obtained by subtracting the filtered image from the original image. Preserves the original residual signal and is used for quantitative analysis. |
|
|
| **`residual_noise_display`** | Visual representation of residual noise after normalization and contrast enhancement. Intended exclusively for qualitative inspection. |
|
|
| **`fft_spectrum`** | Normalized representation of the Fourier spectrum, illustrating the spatial frequency distribution of the image. |
|
|
|
|
---
|
|
|
|
## 6. Feature Summary
|
|
|
|
The following table summarizes the metadata, quantitative features, and explainable maps provided by the library.
|
|
|
|
| Category | Variables |
|
|
|----------|-----------|
|
|
| **Metadata** | `filename`, `target` |
|
|
| **Spectral Features** | `spectral_entropy`, `anisotropy`, `dominant_frequency`, `energy_density`, `high_frequency_ratio` |
|
|
| **Spatial Features** | `mean_brightness`, `mean_darkness`, `mean_gradient`, `mean_local_entropy`, `mean_local_variance` |
|
|
| **Residual Noise Features** | `residual_noise_energy`, `residual_noise_std`, `residual_noise_mean_absolute` |
|
|
| **Explainable Feature Maps** | `brightness_map`, `darkness_map`, `gradient_map`, `local_entropy_map`, `local_variance_map`, `residual_noise_raw`, `residual_noise_display`, `fft_spectrum` |
|
|
|
|
### Summary of Extracted Information
|
|
|
|
The library provides:
|
|
|
|
- **13 quantitative features**, including spectral, spatial, and residual noise descriptors.
|
|
- **8 explainable feature maps** for visual inspection and forensic interpretation.
|
|
- **2 metadata fields** for image identification and class labeling.
|
|
|
|
These outputs support quantitative image characterization, statistical analysis, machine learning workflows, and forensic investigations involving real and synthetically generated images.
|
|
|
|
If a filename contains `gen`, the generated-image target is set to `"0"`. If it contains `real`, the target is set to `"1"`; otherwise, the target is left as `None`.
|
|
|
|
## Interactive visualizations
|
|
|
|
The plotting functions accept dictionaries, pandas `DataFrame` objects, and other data structures supported by `pandas.DataFrame`.
|
|
|
|
```python
|
|
from spectralforensics import line_plot, box_plot
|
|
|
|
data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],}
|
|
line_plot(data, x="sample", y="spectral_entropy")
|
|
box_plot(data, x="spectral_entropy", y="anisotropy")
|
|
```
|
|

|
|

|
|
|
|
Available plotting functions:
|
|
|
|
```python
|
|
from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,)
|
|
```
|
|
### Correlation analysis
|
|
|
|
```python
|
|
from spectralforensics import correlation_plot
|
|
|
|
correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],}
|
|
|
|
correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target", method="pearson", target="target", kind="barh",)
|
|
```
|
|
|
|

|
|
|
|
Set `show=False` to generate and retrieve the Matplotlib figure, axes, and correlation tables without opening the interactive window.
|
|
|
|
## Interactive Chart Controls and Navigation
|
|
|
|
The interactive chart provides a set of controls that allow users to customize data visualization, explore relationships between variables, and interact with the graphical representation.
|
|
|
|
### 1. Axis Configuration and Variable Selection
|
|
|
|
At the top of the chart, two dropdown menus allow users to select the variables displayed on the **X-axis** and **Y-axis**.
|
|
|
|
- **X-axis:** Supports both categorical (qualitative) and numerical (quantitative) variables, enabling different types of data exploration and comparison.
|
|
- **Y-axis:** Accepts only numerical (quantitative) variables, ensuring that the selected data can be represented on a continuous numerical scale.
|
|
|
|
When a **categorical variable** is selected for the X-axis, an additional dropdown menu becomes available, allowing users to select specific categories associated with that variable.
|
|
|
|
Two additional buttons are provided:
|
|
|
|
- **Save Image:** Exports the current chart as an image.
|
|
- **Reset View:** Restores the chart to its initial position and visualization settings.
|
|
|
|
### 2. Interactive Navigation Toolbar
|
|
|
|
An interactive toolbar is located in the **bottom-left corner of the chart**, providing several tools for navigating, adjusting, and exporting the visualization.
|
|
|
|
The toolbar includes the following controls, described **from right to left**:
|
|
|
|
| Control | Function |
|
|
|---------|----------|
|
|
| **Save (Floppy Disk Icon)** | Saves a snapshot of the current chart, allowing users to export the visualization as an image file. |
|
|
| **Configure Subplots** | Allows users to adjust the chart layout, including subplot dimensions, spacing, and positioning. |
|
|
| **Zoom (Magnifying Glass Icon)** | Enables users to zoom into a selected region of the chart for more detailed inspection. |
|
|
| **Pan (Multidirectional Arrows Icon)** | Allows users to move or reposition the chart interactively while maintaining the current zoom level. |
|
|
| **Forward (Right Arrow Icon)** | Navigates to the next view in the chart's navigation history. |
|
|
| **Back (Left Arrow Icon)** | Returns to the previous view in the chart's navigation history. |
|
|
| **Home (House Icon)** | Restores the chart to its original view, resetting the zoom level and axis positioning. |
|
|
|
|
|
|

|
|
|
|

|
|
|
|
### 3. Interactive Data Exploration
|
|
|
|
Together, these controls provide a flexible environment for exploring and interpreting the extracted image features.
|
|
|
|
Users can dynamically select variables, filter categorical data, examine specific regions through zooming and panning, navigate between previous views, and export visualizations for further analysis or documentation.
|
|
|
|
These capabilities support **exploratory data analysis (EDA)**, facilitate the identification of patterns and relationships between extracted features, and improve the interpretability of quantitative image forensic analysis.
|
|
|
|
|
|
## Examples
|
|
|
|
Runnable examples are available in the [`examples`](examples) directory:
|
|
|
|
```bash
|
|
python examples/line_plot.py
|
|
python examples/bar_plot.py
|
|
python examples/scatter_plot.py
|
|
python examples/box_plot.py
|
|
python examples/correlation_plot.py
|
|
```
|
|
|
|
## Project structure
|
|
|
|
```text
|
|
spectralforensics/
|
|
|-- examples/ # Usage examples
|
|
|-- src/
|
|
| `-- spectralforensics/
|
|
| |-- dataset.py # File and folder processing
|
|
| |-- features.py # Feature extraction
|
|
| |-- plotting*.py # Interactive visualization tools
|
|
| `-- assets/ # Plot interface assets
|
|
|-- pyproject.toml # Package metadata and dependencies
|
|
`-- README.md
|
|
```
|
|
|
|
## Research scope
|
|
|
|
SpectralForensics is intended as a research and experimentation toolkit. Extracted descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or AI-generated.
|
|
|
|
## Authors
|
|
|
|
- Jose Garcia
|
|
- Patricio Rodrigues
|
|
- Carlos Grilo
|
|
- Rolando Miragaia
|
|
|
|
## Contributing
|
|
|
|
Issues, suggestions, and contributions are welcome. For substantial changes, open an issue first to discuss the proposed approach and its impact on the public API.
|
|
|
|
## Citation
|
|
|
|
If you use SpectralForensics in academic work, please cite the repository. A formal citation and publication reference will be added as the project matures.
|