This commit is contained in:
2026-10-27 17:35:51 +00:00
parent d986498816
commit 75031c666b
12 changed files with 526 additions and 261 deletions
+244 -180
View File
@@ -1,57 +1,69 @@
# SpectralForensics
SpectralForensics is an open-source Python library for explainable digital image forensics. It extracts interpretable descriptors from the frequency, spatial, and residual-noise domains and provides interactive tools for exploring the resulting data.
SpectralForensics is a Python library for explainable digital image forensics.
It extracts interpretable descriptors from the frequency, spatial, and
residual-noise domains and provides interactive tools for exploring the
resulting data.
The project is being developed to support research into AI-generated image detection, synthetic media analysis, diffusion-model fingerprints, and explainable machine-learning pipelines.
The library contains two complementary extraction paths:
- **RGB extraction** preserves the three color channels, computes
channel-wise descriptors, and aggregates them into image-level values.
- **Grayscale extraction** converts the image to one luminance plane and
reproduces the original SpectralForensics formulation.
The project is being developed to support research into AI-generated image
detection, synthetic-media analysis, diffusion-model fingerprints, and
explainable machine-learning pipelines.
> [!NOTE]
> SpectralForensics is under active development. The public API and extracted features may change before the first stable release.
> SpectralForensics is under active development. The public API and extracted
> features may change before the first stable release.
## Features
### Spectral analysis
- Spectral entropy
- Anisotropy
- Dominant frequency
- Directional anisotropy
- Dominant radial frequency
- Energy density and mean energy
- High-frequency energy ratio
- Normalized FFT spectrum for visualization
- Normalized FFT spectra for visualization
### Spatial analysis
- Mean brightness and darkness
- Gradient magnitude
- Sobel gradient magnitude
- Local entropy
- Local variance
- Normalized visual maps for each spatial descriptor
- Normalized visual maps
### Residual-noise analysis
- Residual-noise energy
- Residual-noise standard deviation
- Global residual standard deviation in RGB mode
- Mean absolute residual noise
- Raw and normalized residual-noise maps
### Dataset and visualization tools
- Feature extraction from a single image or a complete folder
- RGB or grayscale extraction from one image or a complete folder
- Sequential renaming of real and AI-generated image datasets
- Optional CSV export
- Support for JPG, JPEG, PNG, BMP, TIFF, and WebP images
- JPG, JPEG, PNG, BMP, TIFF, and WebP support
- Interactive line, bar, scatter, box, and correlation plots
- Selection and filtering of multiple variables directly in the plot window
- Dynamic variable and category filtering
## Requirements
- Python 3.10 or newer
- A graphical desktop environment for the interactive plotting tools
- A graphical desktop environment for interactive plotting
- Tkinter, normally included with standard Python installations
## Installation
Clone the repository and install it in editable mode:
```bash
git clone https://gittea.kl3z00.com/kl3z/spectralforensics
cd spectralforensics
@@ -60,39 +72,82 @@ python -m pip install -e .
The package is not yet advertised as a stable PyPI release.
## Public extraction API
| Function | Input | Color mode | Default size |
| --- | --- | --- | --- |
| `extract_features` | Image or folder | Selected with `color_mode` | RGB: `1024x1024`; grayscale: `255x255` |
| `extract_rgb_features` | Single image | RGB | `1024x1024` |
| `extract_grayscale_features` | Single image | Grayscale | `255x255` |
`extract_features` is the high-level dataset function. RGB remains its default
mode for backward compatibility with the current library.
## Quick start
### Analyse a single image
### Extract RGB features from one image
```python
from spectralforensics import extract_rgb_features
rgb_features = extract_rgb_features(
"images/sample.jpg",
output="quantitative",
)
print(rgb_features["color_mode"])
print(rgb_features["spectral_entropy"])
print(rgb_features["mean_gradient"])
print(rgb_features["residual_noise_std_global"])
```
### Extract grayscale features from one image
```python
from spectralforensics import extract_grayscale_features
grayscale_features = extract_grayscale_features(
"images/sample.jpg",
output="quantitative",
)
print(grayscale_features["color_mode"])
print(grayscale_features["spectral_entropy"])
print(grayscale_features["mean_gradient"])
print(grayscale_features["residual_noise_std"])
```
Both direct extraction functions return a dictionary containing metadata and
the requested descriptors.
### Process a folder in either mode
```python
from spectralforensics import extract_features
features = extract_features("images/sample.jpg")
rgb_data = extract_features(
"images/",
color_mode="rgb",
output="quantitative",
output_csv="rgb_features.csv",
)
print(features["spectral_entropy"])
print(features["mean_gradient"])
print(features["residual_noise_energy"])
grayscale_data = extract_features(
"images/",
color_mode="grayscale",
output="quantitative",
output_csv="grayscale_features.csv",
)
```
For a single image, `extract_features` returns a dictionary containing the file metadata and the requested descriptors.
For a folder, `extract_features` returns a pandas `DataFrame`. Folder traversal
is non-recursive. Set `show_progress=False` to disable the progress bar.
### Analyse a folder
```python
from spectralforensics import extract_features
data = extract_features("images/",output_csv="spectral_features.csv",output="quantitative",)
print(data.head())
```
When a directory is provided, the function processes supported images in that directory and returns a pandas `DataFrame`. Folder traversal is currently non-recursive.
The optional `image_size` argument uses Pillow's `(width, height)` convention.
If omitted, the mode-specific default shown above is used.
### Prepare image filenames
SpectralForensics can rename all supported images in a folder using the
`real` or `gen` prefixes:
```python
from spectralforensics import rename_generated_images, rename_real_images
@@ -100,13 +155,14 @@ real_files = rename_real_images("dataset/real")
generated_files = rename_generated_images("dataset/generated")
```
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`,
and `gen2.png` patterns. Each function returns a list of the new paths.
Renaming is non-recursive and preserves each image format.
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and
`gen2.png` patterns. Each function returns the new paths. Renaming is
non-recursive, preserves each image format, and uses temporary unique names to
avoid collisions.
## Extraction modes
Use the `output` argument to control which results are calculated and returned:
The `output` argument controls which values are calculated and returned:
| Mode | Result |
| --- | --- |
@@ -114,194 +170,196 @@ Use the `output` argument to control which results are calculated and returned:
| `"spectral"` | Spectral descriptors only |
| `"spatial"` | Spatial descriptors only |
| `"residual"` | Residual-noise descriptors only |
| `"quantitative"` | All numerical descriptors, without visual maps |
| `"visual"` | Visual maps, without quantitative descriptors |
| `"quantitative"` | All numerical descriptors without visual maps |
| `"visual"` | Visual maps without quantitative descriptors |
Example:
Additional extraction parameters:
```python
features = extract_features("images/sample.jpg",image_size=(255, 255),high_freq_threshold=0.65, output="spectral",)
| Parameter | Description | Default |
| --- | --- | --- |
| `high_freq_threshold` | Normalized radial threshold for high-frequency energy | `0.65` |
| `center_exclusion` | Half-width of the square excluded around the FFT center | `5` |
| `image_size` | Resized image dimensions as `(width, height)` | Mode-specific |
## RGB and grayscale behavior
### Grayscale path
`features_grayscale.py` converts the input with `Image.convert("L")`. Every
descriptor is therefore computed from one normalized image
`I(x, y) in [0, 1]`.
### RGB path
`features.py` converts the input with `Image.convert("RGB")`, producing
`I(x, y, c)` for `c in {R, G, B}`. The FFT, Sobel gradient, local entropy,
local variance, and median residual are calculated independently for each
channel.
For a channel-wise descriptor `d_c`, the public image-level RGB descriptor is
the arithmetic mean:
```text
d_RGB = (d_R + d_G + d_B) / 3
```
# Extracted Features and Metadata
Brightness, darkness, gradient, local entropy, local variance, residual energy,
and mean absolute residual are equivalent to averaging over all pixels and all
three channels. When `R = G = B`, the aggregated RGB descriptors match the
grayscale descriptors up to floating-point precision.
The SpectralForensics library extracts a set of quantitative features and interpretable feature maps designed to characterize images in the frequency and spatial domains, including residual noise analysis.
## Extracted metadata and features
## 1. Metadata
### Metadata
| Variable | Description |
|----------|-------------|
| **`filename`** | Name of the processed image file. |
| **`target`** | Image class label (`0 = Real`, `1 = Synthetic`). |
| --- | --- |
| `filename` | Name of the processed image file |
| `path` | Original image path |
| `target` | Convenience label inferred from the filename |
| `color_mode` | `"rgb"` or `"grayscale"` |
---
If a filename contains `gen`, `target` is set to `"0"`. If it contains `real`,
`target` is set to `"1"`. Otherwise it is `None`. This convention is not a
trusted provenance mechanism; research datasets should maintain authoritative
labels separately.
## 2. Spectral Features
These features characterize the frequency-domain properties of an image using Fourier-based analysis.
### Spectral descriptors
| Variable | Description |
|----------|-------------|
| **`spectral_entropy`** | Measures the complexity of the energy distribution in the frequency domain. Higher values indicate a more dispersed spectral energy distribution. |
| **`anisotropy`** | Quantifies the difference between horizontal and vertical spectral energy, enabling the identification of directional frequency patterns. |
| **`dominant_frequency`** | Measures the distance between the highest-energy frequency component and the center of the Fourier spectrum, representing the dominant spatial frequency of the image. |
| **`energy_density`** | Average spectral energy per Fourier transform coefficient. |
| **`high_frequency_ratio`** | Proportion of spectral energy concentrated in high-frequency components relative to the total spectral energy. |
| --- | --- |
| `spectral_entropy` | Shannon entropy of the normalized Fourier power |
| `anisotropy` | Relative difference between horizontal and vertical central-band energy |
| `dominant_frequency` | Radial FFT-bin distance of the maximum-energy component outside the excluded center |
| `energy_density` | Total spectral energy divided by the number of spatial coefficients |
| `high_frequency_ratio` | Fraction of spectral energy beyond the configured radial threshold |
| `mean_energy` | Mean Fourier power |
---
In RGB mode, each value is first computed independently for R, G, and B and
then averaged across channels.
## 3. Spatial Features
These features describe image intensity, local texture, and structural variations in the spatial domain.
### Spatial descriptors
| Variable | Description |
|----------|-------------|
| **`mean_brightness`** | Average pixel intensity of the grayscale image. |
| **`mean_darkness`** | Average value of the darkness map, indicating the prevalence of dark regions within the image. |
| **`mean_gradient`** | Average gradient magnitude, representing the overall presence of edges and intensity transitions. |
| **`mean_local_entropy`** | Average local entropy, reflecting the complexity and texture variability of the image. |
| **`mean_local_variance`** | Average local intensity variance, characterizing local texture variability. |
| --- | --- |
| `mean_brightness` | Mean normalized intensity over pixels and available channels |
| `mean_darkness` | Mean intensity complement; equal to `1 - mean_brightness` |
| `mean_gradient` | Mean channel-wise Sobel-gradient magnitude |
| `mean_local_entropy` | Mean local entropy using a disk footprint of radius 3 |
| `mean_local_variance` | Mean local variance using a `7x7` spatial neighborhood |
---
### Residual-noise descriptors
## 4. Residual Noise Features
| Variable | Modes | Description |
| --- | --- | --- |
| `residual_noise_energy` | Both | Mean squared median-filter residual |
| `residual_noise_std` | Both | Grayscale residual standard deviation, or mean of the three channel standard deviations in RGB |
| `residual_noise_std_global` | RGB only | Standard deviation over all RGB residual values jointly |
| `residual_noise_mean_absolute` | Both | Mean absolute residual magnitude |
These features quantify the statistical properties of residual noise obtained through image filtering.
The median filter uses a `3x3` spatial neighborhood. In RGB mode its channel
size is one, so color channels are never mixed.
| Variable | Description |
|----------|-------------|
| **`residual_noise_energy`** | Mean energy of the residual noise signal. |
| **`residual_noise_std`** | Standard deviation of the residual noise signal. |
| **`residual_noise_mean_absolute`** | Mean absolute value of the residual noise signal. |
## Explainable feature maps
---
| Variable | Grayscale shape | RGB shape | Description |
| --- | --- | --- | --- |
| `fft_spectrum` | `(H, W)` | `(H, W)` | Normalized aggregate logarithmic spectrum |
| `fft_spectrum_channels` | Not returned | `(H, W, 3)` | Independently normalized R, G, and B logarithmic spectra |
| `brightness_map` | `(H, W)` | `(H, W, 3)` | Normalized image intensities |
| `darkness_map` | `(H, W)` | `(H, W, 3)` | Intensity complements |
| `gradient_map` | `(H, W)` | `(H, W)` | Normalized gradient; RGB channels are averaged before display normalization |
| `local_entropy_map` | `(H, W)` | `(H, W)` | Normalized local entropy; RGB channels are averaged |
| `local_variance_map` | `(H, W)` | `(H, W)` | Normalized local variance; RGB channels are averaged |
| `residual_noise_raw` | `(H, W)` | `(H, W, 3)` | Signed median-filter residual |
| `residual_noise` | `(H, W)` | `(H, W, 3)` | Square-root-enhanced residual visualization |
## 5. Explainable Feature Maps
Visual normalization does not alter the quantitative descriptors.
In addition to quantitative features, the library generates interpretable maps that enable visual inspection of spatial structures, texture characteristics, frequency distributions, and residual noise patterns.
## Output summary
| Variable | Description |
|----------|-------------|
| **`brightness_map`** | Normalized luminance intensity map (`0 = Black`, `1 = White`). |
| **`darkness_map`** | Inverted luminance intensity map (`0 = White`, `1 = Black`). |
| **`gradient_map`** | Gradient magnitude map highlighting edges and intensity transitions. |
| **`local_entropy_map`** | Local entropy map highlighting regions with increased texture complexity and information variability. |
| **`local_variance_map`** | Local variance map highlighting regions with greater local intensity variability. |
| **`residual_noise_raw`** | Raw residual noise obtained by subtracting the filtered image from the original image. Preserves the original residual signal and is used for quantitative analysis. |
| **`residual_noise_display`** | Visual representation of residual noise after normalization and contrast enhancement. Intended exclusively for qualitative inspection. |
| **`fft_spectrum`** | Normalized representation of the Fourier spectrum, illustrating the spatial frequency distribution of the image. |
| Mode | Quantitative descriptors | Visual maps |
| --- | ---: | ---: |
| Grayscale | 14 | 8 |
| RGB | 15 | 9 |
---
## 6. Feature Summary
The following table summarizes the metadata, quantitative features, and explainable maps provided by the library.
| Category | Variables |
|----------|-----------|
| **Metadata** | `filename`, `target` |
| **Spectral Features** | `spectral_entropy`, `anisotropy`, `dominant_frequency`, `energy_density`, `high_frequency_ratio` |
| **Spatial Features** | `mean_brightness`, `mean_darkness`, `mean_gradient`, `mean_local_entropy`, `mean_local_variance` |
| **Residual Noise Features** | `residual_noise_energy`, `residual_noise_std`, `residual_noise_mean_absolute` |
| **Explainable Feature Maps** | `brightness_map`, `darkness_map`, `gradient_map`, `local_entropy_map`, `local_variance_map`, `residual_noise_raw`, `residual_noise_display`, `fft_spectrum` |
### Summary of Extracted Information
The library provides:
- **13 quantitative features**, including spectral, spatial, and residual noise descriptors.
- **8 explainable feature maps** for visual inspection and forensic interpretation.
- **2 metadata fields** for image identification and class labeling.
These outputs support quantitative image characterization, statistical analysis, machine learning workflows, and forensic investigations involving real and synthetically generated images.
If a filename contains `gen`, the generated-image target is set to `"0"`. If it contains `real`, the target is set to `"1"`; otherwise, the target is left as `None`.
These counts exclude the four metadata fields.
## Interactive visualizations
The plotting functions accept dictionaries, pandas `DataFrame` objects, and other data structures supported by `pandas.DataFrame`.
The plotting functions accept dictionaries, pandas `DataFrame` objects, and
other inputs supported by `pandas.DataFrame`.
```python
from spectralforensics import line_plot, box_plot
from spectralforensics import box_plot, line_plot
data = {
"sample": [1, 2, 3, 4, 5],
"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],
"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],
}
data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],}
line_plot(data, x="sample", y="spectral_entropy")
box_plot(data, x="spectral_entropy", y="anisotropy")
box_plot(data, x="sample", y="anisotropy")
```
![SpectralForensics line_plot](img/lines2.jpeg)
![SpectralForensics box_plot](img/boxplot.jpeg)
![SpectralForensics line plot](img/lines2.jpg)
![SpectralForensics box plot](img/boxplot.jpg)
Available plotting functions:
```python
from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,)
from spectralforensics import (
bar_plot,
box_plot,
correlation_plot,
line_plot,
scatter_plot,
)
```
### Correlation analysis
```python
from spectralforensics import correlation_plot
correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],}
correlation_data = {
"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],
"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],
"target": [0, 1, 0, 1, 1],
}
correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target", method="pearson", target="target", kind="barh",)
correlation_plot(
correlation_data,
correlation_columns=["spectral_entropy", "anisotropy", "target"],
method="pearson",
target="target",
kind="barh",
)
```
![SpectralForensics box_plot](img/correlation.jpeg)
![SpectralForensics correlation plot](img/correlation.jpg)
Set `show=False` to generate and retrieve the Matplotlib figure, axes, and correlation tables without opening the interactive window.
Set `show=False` to return the Matplotlib figure, axes, and correlation tables
without opening the interactive window.
## Interactive Chart Controls and Navigation
## Interactive chart controls
The interactive chart provides a set of controls that allow users to customize data visualization, explore relationships between variables, and interact with the graphical representation.
The chart interface supports:
### 1. Axis Configuration and Variable Selection
- X-axis and multiple Y-variable selection
- Category filtering
- Zooming and panning
- Navigation through previous views
- Resetting the current view
- Copying or saving a plot
At the top of the chart, two dropdown menus allow users to select the variables displayed on the **X-axis** and **Y-axis**.
- **X-axis:** Supports both categorical (qualitative) and numerical (quantitative) variables, enabling different types of data exploration and comparison.
- **Y-axis:** Accepts only numerical (quantitative) variables, ensuring that the selected data can be represented on a continuous numerical scale.
When a **categorical variable** is selected for the X-axis, an additional dropdown menu becomes available, allowing users to select specific categories associated with that variable.
Two additional buttons are provided:
- **Save Image:** Exports the current chart as an image.
- **Reset View:** Restores the chart to its initial position and visualization settings.
### 2. Interactive Navigation Toolbar
An interactive toolbar is located in the **bottom-left corner of the chart**, providing several tools for navigating, adjusting, and exporting the visualization.
The toolbar includes the following controls, described **from right to left**:
| Control | Function |
|---------|----------|
| **Save (Floppy Disk Icon)** | Saves a snapshot of the current chart, allowing users to export the visualization as an image file. |
| **Configure Subplots** | Allows users to adjust the chart layout, including subplot dimensions, spacing, and positioning. |
| **Zoom (Magnifying Glass Icon)** | Enables users to zoom into a selected region of the chart for more detailed inspection. |
| **Pan (Multidirectional Arrows Icon)** | Allows users to move or reposition the chart interactively while maintaining the current zoom level. |
| **Forward (Right Arrow Icon)** | Navigates to the next view in the chart's navigation history. |
| **Back (Left Arrow Icon)** | Returns to the previous view in the chart's navigation history. |
| **Home (House Icon)** | Restores the chart to its original view, resetting the zoom level and axis positioning. |
![SpectralForensics Top bottons](img/lines.jpeg)
![SpectralForensics Bottom bottons](img/menu.jpeg)
### 3. Interactive Data Exploration
Together, these controls provide a flexible environment for exploring and interpreting the extracted image features.
Users can dynamically select variables, filter categorical data, examine specific regions through zooming and panning, navigate between previous views, and export visualizations for further analysis or documentation.
These capabilities support **exploratory data analysis (EDA)**, facilitate the identification of patterns and relationships between extracted features, and improve the interpretability of quantitative image forensic analysis.
![SpectralForensics top controls](img/lines.jpg)
![SpectralForensics navigation toolbar](img/menu.jpg)
## Examples
Runnable examples are available in the [`examples`](examples) directory:
```bash
python examples/line_plot.py
python examples/bar_plot.py
@@ -314,20 +372,24 @@ python examples/correlation_plot.py
```text
spectralforensics/
|-- examples/ # Usage examples
|-- examples/ # Plotting examples
|-- img/ # README screenshots
|-- src/
| `-- spectralforensics/
| |-- dataset.py # File and folder processing
| |-- features.py # Feature extraction
| |-- plotting*.py # Interactive visualization tools
| `-- assets/ # Plot interface assets
|-- pyproject.toml # Package metadata and dependencies
| |-- dataset.py # File and folder processing
| |-- features.py # RGB feature extraction
| |-- features_grayscale.py # Grayscale feature extraction
| |-- plotting*.py # Interactive visualizations
| `-- assets/ # Interface assets
|-- pyproject.toml # Package metadata and dependencies
`-- README.md
```
## Research scope
SpectralForensics is intended as a research and experimentation toolkit. Extracted descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or AI-generated.
SpectralForensics is a research and experimentation toolkit. Its descriptors
can support statistical analysis and machine-learning models, but they should
not be treated as standalone proof that an image is authentic or generated.
## Authors
@@ -338,8 +400,10 @@ SpectralForensics is intended as a research and experimentation toolkit. Extract
## Contributing
Issues, suggestions, and contributions are welcome. For substantial changes, open an issue first to discuss the proposed approach and its impact on the public API.
Issues, suggestions, and contributions are welcome. For substantial changes,
open an issue first to discuss the approach and its impact on the public API.
## Citation
If you use SpectralForensics in academic work, please cite the repository. A formal citation and publication reference will be added as the project matures.
If you use SpectralForensics in academic work, cite the repository. A formal
citation and publication reference will be added as the project matures.
+2 -1
View File
@@ -27,7 +27,8 @@ dependencies = [
"scipy",
"matplotlib",
"seaborn>=0.13",
"scikit-image"
"scikit-image",
"tqdm"
]
[tool.setuptools]
+4
View File
@@ -3,6 +3,8 @@ from .dataset import (
rename_generated_images,
rename_real_images,
)
from .features import extract_features as extract_rgb_features
from .features_grayscale import extract_grayscale_features
from .plotting import line_plot
from .plotting_bars import bar_plot
from .plotting_scatter import scatter_plot
@@ -11,6 +13,8 @@ from .plotting_box import box_plot
__all__ = [
"extract_features",
"extract_rgb_features",
"extract_grayscale_features",
"rename_real_images",
"rename_generated_images",
"line_plot",
+17 -5
View File
@@ -1,7 +1,8 @@
from pathlib import Path
from uuid import uuid4
import pandas as pd
from .features import extract_features as extract_image_features
from .features import extract_features as extract_rgb_image_features
from .features_grayscale import extract_grayscale_features
from tqdm import tqdm
support_ext = {".jpg", ".jpeg", ".png", ".bmp", ".tif", ".tiff", ".webp"}
@@ -11,7 +12,6 @@ def _rename_images(folder_path, prefix):
raise FileNotFoundError(f"Folder not found: {folder}")
if not folder.is_dir():
raise NotADirectoryError(f"Expected a folder: {folder}")
images = sorted((file for file in folder.iterdir()if file.is_file() and file.suffix.lower() in support_ext),
key=lambda file: file.name.lower(),)
staged_images = []
@@ -33,20 +33,32 @@ def rename_real_images(folder_path):
def rename_generated_images(folder_path):
return _rename_images(folder_path, "gen")
def extract_features(path, output_csv=None, image_size=(255, 255), high_freq_threshold=0.65, output="all", show_progress=True):
def extract_features(path,output_csv=None,image_size=None,high_freq_threshold=0.65,center_exclusion=5,
output="all",show_progress=True, color_mode="rgb",):
path = Path(path)
color_mode = str(color_mode).lower()
extractors = {"rgb": extract_rgb_image_features,"grayscale": extract_grayscale_features,}
if color_mode not in extractors:
raise ValueError("color_mode must be either 'rgb' or 'grayscale'")
extractor = extractors[color_mode]
if image_size is None:
image_size = (1024, 1024) if color_mode == "rgb" else (255, 255)
if not path.exists():
raise FileNotFoundError(f"Path not found: {path}")
if path.is_file():
if path.suffix.lower() not in support_ext:
raise ValueError(f"Unsupported image format: {path.suffix}")
return extract_image_features(path, image_size=image_size, high_freq_threshold=high_freq_threshold, output=output)
return extractor(path,image_size=image_size,high_freq_threshold=high_freq_threshold,center_exclusion=center_exclusion,output=output,)
image_files = [file for file in path.iterdir() if file.is_file() and file.suffix.lower() in support_ext]
iterator = (tqdm(image_files,total=len(image_files),desc="Extracting features",unit="image") if show_progress else image_files)
results = []
for image_file in iterator:
try:
features = extract_image_features(image_file, image_size=image_size, high_freq_threshold=high_freq_threshold, output=output)
features = extractor(image_file,
image_size=image_size,
high_freq_threshold=high_freq_threshold,
center_exclusion=center_exclusion,
output=output,)
results.append(features)
except Exception as e:
results.append({"filename": image_file.name, "path": str(image_file), "error": str(e)})
+127 -75
View File
@@ -5,19 +5,21 @@ from scipy.ndimage import median_filter, sobel, uniform_filter
from skimage.filters.rank import entropy
from skimage.morphology import disk
def extract_features(image_path, image_size=(255, 255), high_freq_threshold=0.65, center_exclusion=5, output="all"):
def extract_features(image_path,image_size=(1024, 1024),high_freq_threshold=0.65,center_exclusion=5,output="all"):
image_path = Path(image_path)
output = str(output).lower()
valid_outputs = {"all", "spectral", "spatial", "residual", "quantitative", "visual"}
valid_outputs = {"all", "spectral", "spatial","residual", "quantitative", "visual"}
if output not in valid_outputs:
raise ValueError("output must be one of: 'all', 'spectral', 'spatial', 'residual', 'quantitative', 'visual'")
raise ValueError("output must be one of: 'all', 'spectral', ""'spatial', 'residual', 'quantitative', 'visual'")
if not image_path.exists():
raise FileNotFoundError(f"Image not found: {image_path}")
if not image_path.is_file():
raise ValueError(f"Expected an image file: {image_path}")
image = Image.open(image_path).convert("L")
image = image.resize(image_size)
img_array = np.asarray(image, dtype=np.float32) / np.float32(255.0)
with Image.open(image_path) as image:
image = image.convert("RGB")
image = image.resize(image_size)
img_array = np.asarray(image, dtype=np.float32) / np.float32(255.0)
height, width, channels = img_array.shape
filename_lower = image_path.name.lower()
if "gen" in filename_lower:
target = "0"
@@ -25,7 +27,7 @@ def extract_features(image_path, image_size=(255, 255), high_freq_threshold=0.65
target = "1"
else:
target = None
results = {"filename": image_path.name,"path": str(image_path),"target": target}
results = {"filename": image_path.name,"path": str(image_path),"target": target,"color_mode": "rgb",}
calculate_spectral = output in {"all", "spectral", "quantitative", "visual"}
calculate_spatial = output in {"all", "spatial", "quantitative", "visual"}
calculate_residual = output in {"all", "residual", "quantitative", "visual"}
@@ -33,53 +35,74 @@ def extract_features(image_path, image_size=(255, 255), high_freq_threshold=0.65
return_spatial_values = output in {"all", "spatial", "quantitative"}
return_residual_values = output in {"all", "residual", "quantitative"}
return_visual_maps = output in {"all", "visual"}
eps = 1e-12
if calculate_spectral:
fft = np.fft.fft2(img_array)
fft_shift = np.fft.fftshift(fft)
fft = np.fft.fft2(img_array, axes=(0, 1))
fft_shift = np.fft.fftshift(fft, axes=(0, 1))
magnitude = np.abs(fft_shift).astype(np.float32)
energy = np.square(magnitude, dtype=np.float32)
total_energy = float(np.sum(energy, dtype=np.float64)) + 1e-12
height, width = energy.shape
center_y = height // 2
center_x = width // 2
y_indices, x_indices = np.indices((height, width), dtype=np.float32)
radius = np.sqrt((x_indices - center_x) ** 2 + (y_indices - center_y) ** 2).astype(np.float32)
maximum_radius = np.max(radius)
high_frequency_mask = (radius > high_freq_threshold * maximum_radius)
if return_spectral_values:
probability = energy / total_energy
spectral_entropy = -np.sum(probability * np.log2(probability + 1e-12), dtype=np.float64)
horizontal_energy = np.sum(energy[max(0, center_y - 2):min(height, center_y + 3), :], dtype=np.float64)
vertical_energy = np.sum(energy[:, max(0, center_x - 2):min(width, center_x + 3)], dtype=np.float64)
anisotropy = abs(horizontal_energy - vertical_energy) / (horizontal_energy + vertical_energy + 1e-12)
energy_no_center = energy.copy()
y_start = max(0, center_y - center_exclusion)
y_end = min(height, center_y + center_exclusion + 1)
x_start = max(0, center_x - center_exclusion)
x_end = min(width, center_x + center_exclusion + 1)
energy_no_center[y_start:y_end, x_start:x_end] = 0
dominant_position = np.unravel_index(np.argmax(energy_no_center), energy_no_center.shape)
dominant_frequency = radius[dominant_position]
energy_density = total_energy / (height * width)
maximum_radius = np.max(radius)
high_frequency_mask = radius > high_freq_threshold * maximum_radius
high_frequency_energy = np.sum(energy[high_frequency_mask], dtype=np.float64)
high_frequency_ratio = high_frequency_energy / total_energy
mean_energy = np.mean(energy, dtype=np.float64)
results.update({
"spectral_entropy": float(spectral_entropy),
"anisotropy": float(anisotropy),
"dominant_frequency": float(dominant_frequency),
"energy_density": float(energy_density),
"high_frequency_ratio": float(high_frequency_ratio),
"mean_energy": float(mean_energy)})
del probability
del energy_no_center
spectral_entropy_channels = []
anisotropy_channels = []
dominant_frequency_channels = []
energy_density_channels = []
high_frequency_ratio_channels = []
mean_energy_channels = []
for c in range(channels):
energy_c = energy[:, :, c]
total_energy_c = float(np.sum(energy_c, dtype=np.float64)) + eps
probability_c = energy_c / total_energy_c
spectral_entropy_c = -np.sum(probability_c * np.log2(probability_c + eps),dtype=np.float64)
horizontal_energy_c = np.sum(energy_c[max(0, center_y - 2):
min(height, center_y + 3), :],dtype=np.float64)
vertical_energy_c = np.sum(energy_c[:,max(0, center_x - 2):
min(width, center_x + 3)],dtype=np.float64)
anisotropy_c = (abs(horizontal_energy_c - vertical_energy_c) / (horizontal_energy_c +vertical_energy_c + eps))
energy_no_center = energy_c.copy()
y_start = max(0, center_y - center_exclusion)
y_end = min(height, center_y + center_exclusion + 1)
x_start = max(0, center_x - center_exclusion)
x_end = min(width, center_x + center_exclusion + 1)
energy_no_center[y_start:y_end,x_start:x_end] = 0
dominant_position = np.unravel_index(np.argmax(energy_no_center),energy_no_center.shape)
dominant_frequency_c = float(radius[dominant_position])
energy_density_c = (total_energy_c / (height * width))
high_frequency_energy_c = np.sum(energy_c[high_frequency_mask],dtype=np.float64)
high_frequency_ratio_c = (high_frequency_energy_c /total_energy_c)
mean_energy_c = np.mean(energy_c, dtype=np.float64)
spectral_entropy_channels.append(spectral_entropy_c)
anisotropy_channels.append(anisotropy_c)
dominant_frequency_channels.append(dominant_frequency_c)
energy_density_channels.append(energy_density_c)
high_frequency_ratio_channels.append(high_frequency_ratio_c)
mean_energy_channels.append(mean_energy_c)
results.update({"spectral_entropy": float(np.mean(spectral_entropy_channels)),
"anisotropy": float(np.mean(anisotropy_channels)),
"dominant_frequency": float(np.mean(dominant_frequency_channels)),
"energy_density": float(np.mean(energy_density_channels)),
"high_frequency_ratio": float(np.mean(high_frequency_ratio_channels)),
"mean_energy": float(np.mean(mean_energy_channels))})
if return_visual_maps:
fft_spectrum_raw = np.log1p(magnitude)
fft_minimum = fft_spectrum_raw.min()
fft_maximum = fft_spectrum_raw.max()
fft_spectrum = (255.0 * (fft_spectrum_raw - fft_minimum) / (fft_maximum - fft_minimum + 1e-12)).astype(np.uint8)
results["fft_spectrum"] = fft_spectrum
del fft_spectrum_raw
fft_spectrum_channels = np.empty((height, width, channels),dtype=np.uint8)
for c in range(channels):
spectrum_c = fft_spectrum_raw[:, :, c]
spectrum_min = spectrum_c.min()
spectrum_max = spectrum_c.max()
fft_spectrum_channels[:, :, c] = (255.0 *(spectrum_c - spectrum_min) / (spectrum_max - spectrum_min + eps)).astype(np.uint8)
aggregated_energy = np.mean(energy, axis=2, dtype=np.float64)
aggregated_magnitude = np.sqrt(aggregated_energy)
aggregated_spectrum_raw = np.log1p(aggregated_magnitude)
spectrum_min = aggregated_spectrum_raw.min()
spectrum_max = aggregated_spectrum_raw.max()
fft_spectrum = (255.0 *(aggregated_spectrum_raw - spectrum_min) / (spectrum_max - spectrum_min + eps)).astype(np.uint8)
results.update({"fft_spectrum_channels": fft_spectrum_channels,"fft_spectrum": fft_spectrum})
del fft
del fft_shift
del magnitude
@@ -87,35 +110,55 @@ def extract_features(image_path, image_size=(255, 255), high_freq_threshold=0.65
del radius
del y_indices
del x_indices
if calculate_spatial:
gradient_x = sobel(img_array, axis=1, mode="reflect").astype(np.float32)
gradient_y = sobel(img_array, axis=0, mode="reflect").astype(np.float32)
gradient_x = np.empty_like(img_array)
gradient_y = np.empty_like(img_array)
for c in range(channels):
gradient_x[:, :, c] = sobel(
img_array[:, :, c], axis=1, mode="reflect"
)
gradient_y[:, :, c] = sobel(
img_array[:, :, c], axis=0, mode="reflect"
)
gradient_raw = np.sqrt(gradient_x ** 2 + gradient_y ** 2).astype(np.float32)
local_mean = uniform_filter(img_array, size=7, mode="reflect").astype(np.float32)
local_squared_mean = uniform_filter(np.square(img_array, dtype=np.float32), size=7, mode="reflect").astype(np.float32)
local_variance_raw = np.maximum(local_squared_mean - local_mean ** 2, 0).astype(np.float32)
image_uint8 = (img_array * 255).astype(np.uint8)
local_entropy_raw = entropy(image_uint8, footprint=disk(3)).astype(np.float32)
local_mean = uniform_filter(img_array,size=(7, 7, 1),mode="reflect").astype(np.float32)
local_squared_mean = uniform_filter(np.square(img_array, dtype=np.float32),size=(7, 7, 1),mode="reflect").astype(np.float32)
local_variance_raw = np.maximum(local_squared_mean - local_mean ** 2,0).astype(np.float32)
image_uint8 = np.asarray(np.clip(np.rint(img_array * 255.0), 0, 255),dtype=np.uint8)
local_entropy_raw = np.empty((height, width, channels),dtype=np.float32)
entropy_footprint = disk(3)
for c in range(channels):
local_entropy_raw[:, :, c] = entropy(image_uint8[:, :, c],footprint=entropy_footprint).astype(np.float32)
if return_spatial_values:
results.update({
"mean_brightness": float(np.mean(img_array, dtype=np.float64)),
results.update({"mean_brightness": float(np.mean(img_array, dtype=np.float64)),
"mean_darkness": float(np.mean(1.0 - img_array, dtype=np.float64)),
"mean_gradient": float(np.mean(gradient_raw, dtype=np.float64)),
"mean_local_entropy": float(np.mean(local_entropy_raw, dtype=np.float64)),
"mean_local_variance": float(np.mean(local_variance_raw, dtype=np.float64))})
if return_visual_maps:
brightness_map = img_array.copy()
darkness_map = (1.0 - img_array).astype(np.float32)
gradient_minimum = gradient_raw.min()
gradient_maximum = gradient_raw.max()
gradient_map = ((gradient_raw - gradient_minimum) / (gradient_maximum - gradient_minimum + 1e-12)).astype(np.float32)
entropy_minimum = local_entropy_raw.min()
entropy_maximum = local_entropy_raw.max()
local_entropy_map = ((local_entropy_raw - entropy_minimum) / (entropy_maximum - entropy_minimum + 1e-12)).astype(np.float32)
variance_minimum = local_variance_raw.min()
variance_maximum = local_variance_raw.max()
local_variance_map = ((local_variance_raw - variance_minimum) / (variance_maximum - variance_minimum + 1e-12)).astype(np.float32)
results.update({"brightness_map": brightness_map,"darkness_map": darkness_map,"gradient_map": gradient_map,"local_entropy_map": local_entropy_map,"local_variance_map": local_variance_map})
gradient_aggregated = np.mean(gradient_raw, axis=2)
entropy_aggregated = np.mean(local_entropy_raw, axis=2)
variance_aggregated = np.mean(local_variance_raw, axis=2)
gradient_minimum = gradient_aggregated.min()
gradient_maximum = gradient_aggregated.max()
gradient_map = ((gradient_aggregated - gradient_minimum) /(gradient_maximum - gradient_minimum + eps)).astype(np.float32)
entropy_minimum = entropy_aggregated.min()
entropy_maximum = entropy_aggregated.max()
local_entropy_map = ((entropy_aggregated - entropy_minimum) /(entropy_maximum - entropy_minimum + eps)).astype(np.float32)
variance_minimum = variance_aggregated.min()
variance_maximum = variance_aggregated.max()
local_variance_map = ((variance_aggregated - variance_minimum) /(variance_maximum - variance_minimum + eps)).astype(np.float32)
results.update({
"brightness_map": brightness_map,
"darkness_map": darkness_map,
"gradient_map": gradient_map,
"local_entropy_map": local_entropy_map,
"local_variance_map": local_variance_map })
del gradient_x
del gradient_y
del gradient_raw
@@ -124,24 +167,33 @@ def extract_features(image_path, image_size=(255, 255), high_freq_threshold=0.65
del local_variance_raw
del local_entropy_raw
del image_uint8
if calculate_residual:
denoised_image = median_filter(img_array, size=3).astype(np.float32)
denoised_image = median_filter(img_array,size=(3, 3, 1),mode="reflect").astype(np.float32)
residual_noise_raw = (img_array - denoised_image).astype(np.float32)
if return_residual_values:
results.update({
"residual_noise_energy": float(np.mean(np.square(residual_noise_raw, dtype=np.float32), dtype=np.float64)),
"residual_noise_std": float(np.std(residual_noise_raw, dtype=np.float64)),
"residual_noise_mean_absolute": float(np.mean(np.abs(residual_noise_raw), dtype=np.float64))})
residual_noise_energy = np.mean(
np.square(residual_noise_raw,dtype=np.float32),dtype=np.float64)
residual_std_channels = np.std(residual_noise_raw,axis=(0, 1),dtype=np.float64)
residual_noise_std = np.mean(residual_std_channels)
residual_noise_std_global = np.std(residual_noise_raw,dtype=np.float64)
residual_noise_mean_absolute = np.mean(np.abs(residual_noise_raw),dtype=np.float64)
results.update({"residual_noise_energy": float(residual_noise_energy),
"residual_noise_std": float(residual_noise_std),
"residual_noise_std_global": float(residual_noise_std_global),
"residual_noise_mean_absolute": float(residual_noise_mean_absolute)})
if return_visual_maps:
residual_absolute = np.abs(residual_noise_raw).astype(np.float32)
residual_minimum = residual_absolute.min()
residual_maximum = residual_absolute.max()
residual_noise_display = np.sqrt((residual_absolute - residual_minimum) / (residual_maximum - residual_minimum + 1e-12)).astype(np.float32)
results.update({
"residual_noise_raw": residual_noise_raw,
"residual_noise": residual_noise_display})
residual_noise_display = np.empty_like(residual_absolute)
for c in range(channels):
residual_c = residual_absolute[:, :, c]
residual_minimum = residual_c.min()
residual_maximum = residual_c.max()
residual_noise_display[:, :, c] = np.sqrt((residual_c - residual_minimum) / (residual_maximum - residual_minimum + eps)).astype(np.float32)
results.update({"residual_noise_raw": residual_noise_raw,"residual_noise": residual_noise_display})
del residual_absolute
del denoised_image
if not return_visual_maps:
del residual_noise_raw
return results
return results
+132
View File
@@ -0,0 +1,132 @@
from pathlib import Path
import numpy as np
from PIL import Image
from scipy.ndimage import median_filter, sobel, uniform_filter
from skimage.filters.rank import entropy
from skimage.morphology import disk
def extract_grayscale_features(image_path,image_size=(255, 255),high_freq_threshold=0.65,center_exclusion=5,output="all",):
image_path = Path(image_path)
output = str(output).lower()
valid_outputs = {"all","spectral","spatial","residual","quantitative","visual",}
if output not in valid_outputs:
raise ValueError("output must be one of: 'all', 'spectral', 'spatial', 'residual', 'quantitative', 'visual'")
if not image_path.exists():
raise FileNotFoundError(f"Image not found: {image_path}")
if not image_path.is_file():
raise ValueError(f"Expected an image file: {image_path}")
with Image.open(image_path) as image:
image = image.convert("L")
image = image.resize(image_size)
img_array = np.asarray(image, dtype=np.float32) / np.float32(255.0)
filename_lower = image_path.name.lower()
if "gen" in filename_lower:
target = "0"
elif "real" in filename_lower:
target = "1"
else:
target = None
results = {"filename": image_path.name,"path": str(image_path),"target": target,"color_mode": "grayscale",}
calculate_spectral = output in {"all", "spectral", "quantitative", "visual"}
calculate_spatial = output in {"all", "spatial", "quantitative", "visual"}
calculate_residual = output in {"all", "residual", "quantitative", "visual"}
return_spectral_values = output in {"all", "spectral", "quantitative"}
return_spatial_values = output in {"all", "spatial", "quantitative"}
return_residual_values = output in {"all", "residual", "quantitative"}
return_visual_maps = output in {"all", "visual"}
eps = 1e-12
if calculate_spectral:
fft = np.fft.fft2(img_array)
fft_shift = np.fft.fftshift(fft)
magnitude = np.abs(fft_shift).astype(np.float32)
energy = np.square(magnitude, dtype=np.float32)
total_energy = float(np.sum(energy, dtype=np.float64)) + eps
height, width = energy.shape
center_y = height // 2
center_x = width // 2
y_indices, x_indices = np.indices((height, width), dtype=np.float32)
radius = np.sqrt((x_indices - center_x) ** 2 + (y_indices - center_y) ** 2).astype(np.float32)
if return_spectral_values:
probability = energy / total_energy
spectral_entropy = -np.sum(probability * np.log2(probability + eps), dtype=np.float64)
horizontal_energy = np.sum(energy[max(0, center_y - 2) : min(height, center_y + 3), :],dtype=np.float64,)
vertical_energy = np.sum(energy[:, max(0, center_x - 2) : min(width, center_x + 3)],dtype=np.float64,)
anisotropy = abs(horizontal_energy - vertical_energy) / (horizontal_energy + vertical_energy + eps)
energy_no_center = energy.copy()
y_start = max(0, center_y - center_exclusion)
y_end = min(height, center_y + center_exclusion + 1)
x_start = max(0, center_x - center_exclusion)
x_end = min(width, center_x + center_exclusion + 1)
energy_no_center[y_start:y_end, x_start:x_end] = 0
dominant_position = np.unravel_index(np.argmax(energy_no_center), energy_no_center.shape)
dominant_frequency = radius[dominant_position]
energy_density = total_energy / (height * width)
maximum_radius = np.max(radius)
high_frequency_mask = radius > high_freq_threshold * maximum_radius
high_frequency_energy = np.sum(energy[high_frequency_mask], dtype=np.float64)
high_frequency_ratio = high_frequency_energy / total_energy
mean_energy = np.mean(energy, dtype=np.float64)
results.update({"spectral_entropy": float(spectral_entropy),
"anisotropy": float(anisotropy),
"dominant_frequency": float(dominant_frequency),
"energy_density": float(energy_density),
"high_frequency_ratio": float(high_frequency_ratio),
"mean_energy": float(mean_energy),})
if return_visual_maps:
fft_spectrum_raw = np.log1p(magnitude)
fft_minimum = fft_spectrum_raw.min()
fft_maximum = fft_spectrum_raw.max()
fft_spectrum = (255.0* (fft_spectrum_raw - fft_minimum)/ (fft_maximum - fft_minimum + eps)).astype(np.uint8)
results["fft_spectrum"] = fft_spectrum
if calculate_spatial:
gradient_x = sobel(img_array, axis=1, mode="reflect").astype(np.float32)
gradient_y = sobel(img_array, axis=0, mode="reflect").astype(np.float32)
gradient_raw = np.sqrt(gradient_x**2 + gradient_y**2).astype(np.float32)
local_mean = uniform_filter(img_array, size=7, mode="reflect").astype(np.float32)
local_squared_mean = uniform_filter(np.square(img_array, dtype=np.float32), size=7, mode="reflect").astype(np.float32)
local_variance_raw = np.maximum(local_squared_mean - local_mean**2, 0).astype(np.float32)
image_uint8 = (img_array * 255).astype(np.uint8)
local_entropy_raw = entropy(image_uint8, footprint=disk(3)).astype(np.float32)
if return_spatial_values:
results.update({"mean_brightness": float(np.mean(img_array, dtype=np.float64)),
"mean_darkness": float(np.mean(1.0 - img_array, dtype=np.float64)),
"mean_gradient": float(np.mean(gradient_raw, dtype=np.float64)),
"mean_local_entropy": float(np.mean(local_entropy_raw, dtype=np.float64)),
"mean_local_variance": float(np.mean(local_variance_raw, dtype=np.float64)),})
if return_visual_maps:
brightness_map = img_array.copy()
darkness_map = (1.0 - img_array).astype(np.float32)
gradient_minimum = gradient_raw.min()
gradient_maximum = gradient_raw.max()
gradient_map = ((gradient_raw - gradient_minimum)/ (gradient_maximum - gradient_minimum + eps)).astype(np.float32)
entropy_minimum = local_entropy_raw.min()
entropy_maximum = local_entropy_raw.max()
local_entropy_map = ((local_entropy_raw - entropy_minimum)/ (entropy_maximum - entropy_minimum + eps)).astype(np.float32)
variance_minimum = local_variance_raw.min()
variance_maximum = local_variance_raw.max()
local_variance_map = ((local_variance_raw - variance_minimum)/ (variance_maximum - variance_minimum + eps)).astype(np.float32)
results.update({"brightness_map": brightness_map,
"darkness_map": darkness_map,
"gradient_map": gradient_map,
"local_entropy_map": local_entropy_map,
"local_variance_map": local_variance_map,})
if calculate_residual:
denoised_image = median_filter(img_array, size=3, mode="reflect").astype(np.float32)
residual_noise_raw = (img_array - denoised_image).astype(np.float32)
if return_residual_values:
results.update({"residual_noise_energy": float(np.mean(
np.square(residual_noise_raw, dtype=np.float32),
dtype=np.float64,)),
"residual_noise_std": float(np.std(residual_noise_raw, dtype=np.float64)),
"residual_noise_mean_absolute": float(np.mean(np.abs(residual_noise_raw), dtype=np.float64)),})
if return_visual_maps:
residual_absolute = np.abs(residual_noise_raw).astype(np.float32)
residual_minimum = residual_absolute.min()
residual_maximum = residual_absolute.max()
residual_noise_display = np.sqrt((residual_absolute - residual_minimum)/ (residual_maximum - residual_minimum + eps)).astype(np.float32)
results.update({"residual_noise_raw": residual_noise_raw,"residual_noise": residual_noise_display,})
return results