Compare commits
8 Commits
a615dbc867
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 29e6355413 | |||
| 75031c666b | |||
| d986498816 | |||
| f16e71c72b | |||
| 5deb660813 | |||
| 06b8812a2b | |||
| 8b91a01e60 | |||
| 861242b3e7 |
@@ -1,292 +1,364 @@
|
||||
SpectralForensics
|
||||
# SpectralForensics
|
||||
|
||||
SpectralForensics is a Python library for explainable digital image forensics.
|
||||
It extracts interpretable descriptors from the frequency, spatial, and
|
||||
residual-noise domains and provides interactive tools for exploring the
|
||||
resulting data.
|
||||
|
||||
The library contains two complementary extraction paths:
|
||||
|
||||
SpectralForensics is an open-source Python library for image forensic analysis using Fast Fourier Transform (FFT) and frequency-domain feature extraction.
|
||||
- **RGB extraction** preserves the three color channels, computes
|
||||
channel-wise descriptors, and aggregates them into image-level values.
|
||||
- **Grayscale extraction** converts the image to one luminance plane and
|
||||
reproduces the original SpectralForensics formulation.
|
||||
|
||||
The project is being developed to support research into AI-generated image
|
||||
detection, synthetic-media analysis, diffusion-model fingerprints, and
|
||||
explainable machine-learning pipelines.
|
||||
|
||||
> [!NOTE]
|
||||
> SpectralForensics is under active development. The public API and extracted
|
||||
> features may change before the first stable release.
|
||||
|
||||
The library was designed to support research in digital image forensics, synthetic media detection, diffusion model analysis, and explainable artificial intelligence (XAI).
|
||||
## Features
|
||||
|
||||
### Spectral analysis
|
||||
|
||||
- Spectral entropy
|
||||
- Directional anisotropy
|
||||
- Dominant radial frequency
|
||||
- Energy density and mean energy
|
||||
- High-frequency energy ratio
|
||||
- Normalized FFT spectra for visualization
|
||||
|
||||
Features
|
||||
### Spatial analysis
|
||||
|
||||
- Mean brightness and darkness
|
||||
- Sobel gradient magnitude
|
||||
- Local entropy
|
||||
- Local variance
|
||||
- Normalized visual maps
|
||||
|
||||
### Residual-noise analysis
|
||||
|
||||
SpectralForensics extracts a set of interpretable spectral descriptors from images:
|
||||
- Residual-noise energy
|
||||
- Residual-noise standard deviation
|
||||
- Global residual standard deviation in RGB mode
|
||||
- Mean absolute residual noise
|
||||
- Raw and normalized residual-noise maps
|
||||
|
||||
### Dataset and visualization tools
|
||||
|
||||
- RGB or grayscale extraction from one image or a complete folder
|
||||
- Sequential renaming of real and AI-generated image datasets
|
||||
- Optional CSV export
|
||||
- JPG, JPEG, PNG, BMP, TIFF, and WebP support
|
||||
- Interactive line, bar, scatter, box, and correlation plots
|
||||
- Dynamic variable and category filtering
|
||||
|
||||
Spectral Entropy
|
||||
## Requirements
|
||||
|
||||
Anisotropy
|
||||
- Python 3.10 or newer
|
||||
- A graphical desktop environment for interactive plotting
|
||||
- Tkinter, normally included with standard Python installations
|
||||
|
||||
Dominant Frequency
|
||||
## Installation
|
||||
|
||||
Energy Density
|
||||
```bash
|
||||
git clone https://gittea.kl3z00.com/kl3z/spectralforensics
|
||||
cd spectralforensics
|
||||
python -m pip install -e .
|
||||
```
|
||||
|
||||
High Frequency Ratio
|
||||
The package is not yet advertised as a stable PyPI release.
|
||||
|
||||
Mean Spectral Energy
|
||||
## Public extraction API
|
||||
|
||||
| Function | Input | Color mode | Default size |
|
||||
| --- | --- | --- | --- |
|
||||
| `extract_features` | Image or folder | Selected with `color_mode` | RGB: `1024x1024`; grayscale: `255x255` |
|
||||
| `extract_rgb_features` | Single image | RGB | `1024x1024` |
|
||||
| `extract_grayscale_features` | Single image | Grayscale | `255x255` |
|
||||
|
||||
`extract_features` is the high-level dataset function. RGB remains its default
|
||||
mode for backward compatibility with the current library.
|
||||
|
||||
These descriptors can be used for:
|
||||
## Quick start
|
||||
|
||||
### Extract RGB features from one image
|
||||
|
||||
```python
|
||||
from spectralforensics import extract_rgb_features
|
||||
|
||||
AI-generated image detection
|
||||
rgb_features = extract_rgb_features("images/sample.jpg",output="quantitative",)
|
||||
print(rgb_features["color_mode"])
|
||||
print(rgb_features["spectral_entropy"])
|
||||
print(rgb_features["mean_gradient"])
|
||||
print(rgb_features["residual_noise_std_global"])
|
||||
```
|
||||
|
||||
Deepfake forensics
|
||||
### Extract grayscale features from one image
|
||||
|
||||
Diffusion model fingerprinting
|
||||
```python
|
||||
from spectralforensics import extract_grayscale_features
|
||||
|
||||
Frequency-domain analysis
|
||||
grayscale_features = extract_grayscale_features("images/sample.jpg",output="quantitative",)
|
||||
print(grayscale_features["color_mode"])
|
||||
print(grayscale_features["spectral_entropy"])
|
||||
print(grayscale_features["mean_gradient"])
|
||||
print(grayscale_features["residual_noise_std"])
|
||||
```
|
||||
|
||||
Explainable machine learning pipelines
|
||||
Both direct extraction functions return a dictionary containing metadata and
|
||||
the requested descriptors.
|
||||
|
||||
Installation
|
||||
### Process a folder in either mode
|
||||
|
||||
Local installation
|
||||
```python
|
||||
from spectralforensics import extract_features
|
||||
|
||||
py -m pip install -e .
|
||||
rgb_data = extract_features("images/",color_mode="rgb",output="quantitative",output_csv="rgb_features.csv",)
|
||||
grayscale_data = extract_features("images/",color_mode="grayscale",output="quantitative",output_csv="grayscale_features.csv",)
|
||||
```
|
||||
|
||||
Future PyPI installation
|
||||
For a folder, `extract_features` returns a pandas `DataFrame`. Folder traversal
|
||||
is non-recursive. Set `show_progress=False` to disable the progress bar.
|
||||
|
||||
pip install spectralforensics
|
||||
The optional `image_size` argument uses Pillow's `(width, height)` convention.
|
||||
If omitted, the mode-specific default shown above is used.
|
||||
|
||||
Supported Image Formats
|
||||
### Prepare image filenames
|
||||
|
||||
```python
|
||||
from spectralforensics import rename_generated_images, rename_real_images
|
||||
|
||||
real_files = rename_real_images("dataset/real")
|
||||
generated_files = rename_generated_images("dataset/generated")
|
||||
```
|
||||
|
||||
The library currently supports:
|
||||
The resulting filenames follow the `real1.jpg`, `real2.png`, `gen1.jpg`, and
|
||||
`gen2.png` patterns. Each function returns the new paths. Renaming is
|
||||
non-recursive, preserves each image format, and uses temporary unique names to
|
||||
avoid collisions.
|
||||
|
||||
## Extraction modes
|
||||
|
||||
The `output` argument controls which values are calculated and returned:
|
||||
|
||||
JPG
|
||||
| Mode | Result |
|
||||
| --- | --- |
|
||||
| `"all"` | All quantitative descriptors and visual maps |
|
||||
| `"spectral"` | Spectral descriptors only |
|
||||
| `"spatial"` | Spatial descriptors only |
|
||||
| `"residual"` | Residual-noise descriptors only |
|
||||
| `"quantitative"` | All numerical descriptors without visual maps |
|
||||
| `"visual"` | Visual maps without quantitative descriptors |
|
||||
|
||||
JPEG
|
||||
Additional extraction parameters:
|
||||
|
||||
PNG
|
||||
| Parameter | Description | Default |
|
||||
| --- | --- | --- |
|
||||
| `high_freq_threshold` | Normalized radial threshold for high-frequency energy | `0.65` |
|
||||
| `center_exclusion` | Half-width of the square excluded around the FFT center | `5` |
|
||||
| `image_size` | Resized image dimensions as `(width, height)` | Mode-specific |
|
||||
|
||||
BMP
|
||||
## RGB and grayscale behavior
|
||||
|
||||
TIFF
|
||||
### Grayscale path
|
||||
|
||||
WEBP
|
||||
`features_grayscale.py` converts the input with `Image.convert("L")`. Every
|
||||
descriptor is therefore computed from one normalized image
|
||||
`I(x, y) in [0, 1]`.
|
||||
|
||||
### RGB path
|
||||
|
||||
`features.py` converts the input with `Image.convert("RGB")`, producing
|
||||
`I(x, y, c)` for `c in {R, G, B}`. The FFT, Sobel gradient, local entropy,
|
||||
local variance, and median residual are calculated independently for each
|
||||
channel.
|
||||
|
||||
Additional formats can easily be added.
|
||||
For a channel-wise descriptor `d_c`, the public image-level RGB descriptor is
|
||||
the arithmetic mean:
|
||||
|
||||
```text
|
||||
d_RGB = (d_R + d_G + d_B) / 3
|
||||
```
|
||||
|
||||
Brightness, darkness, gradient, local entropy, local variance, residual energy,
|
||||
and mean absolute residual are equivalent to averaging over all pixels and all
|
||||
three channels. When `R = G = B`, the aggregated RGB descriptors match the
|
||||
grayscale descriptors up to floating-point precision.
|
||||
|
||||
Quick Start
|
||||
## Extracted metadata and features
|
||||
|
||||
Extract features from a single image
|
||||
### Metadata
|
||||
|
||||
from spectralforensics import extract\_spectral\_features
|
||||
| Variable | Description |
|
||||
| --- | --- |
|
||||
| `filename` | Name of the processed image file |
|
||||
| `path` | Original image path |
|
||||
| `target` | Convenience label inferred from the filename |
|
||||
| `color_mode` | `"rgb"` or `"grayscale"` |
|
||||
|
||||
If a filename contains `gen`, `target` is set to `"0"`. If it contains `real`,
|
||||
`target` is set to `"1"`. Otherwise it is `None`. This convention is not a
|
||||
trusted provenance mechanism; research datasets should maintain authoritative
|
||||
labels separately.
|
||||
|
||||
### Spectral descriptors
|
||||
|
||||
features = extract\_spectral\_features(
|
||||
| Variable | Description |
|
||||
| --- | --- |
|
||||
| `spectral_entropy` | Shannon entropy of the normalized Fourier power |
|
||||
| `anisotropy` | Relative difference between horizontal and vertical central-band energy |
|
||||
| `dominant_frequency` | Radial FFT-bin distance of the maximum-energy component outside the excluded center |
|
||||
| `energy_density` | Total spectral energy divided by the number of spatial coefficients |
|
||||
| `high_frequency_ratio` | Fraction of spectral energy beyond the configured radial threshold |
|
||||
| `mean_energy` | Mean Fourier power |
|
||||
|
||||
  "image.jpg"
|
||||
In RGB mode, each value is first computed independently for R, G, and B and
|
||||
then averaged across channels.
|
||||
|
||||
)
|
||||
### Spatial descriptors
|
||||
|
||||
| Variable | Description |
|
||||
| --- | --- |
|
||||
| `mean_brightness` | Mean normalized intensity over pixels and available channels |
|
||||
| `mean_darkness` | Mean intensity complement; equal to `1 - mean_brightness` |
|
||||
| `mean_gradient` | Mean channel-wise Sobel-gradient magnitude |
|
||||
| `mean_local_entropy` | Mean local entropy using a disk footprint of radius 3 |
|
||||
| `mean_local_variance` | Mean local variance using a `7x7` spatial neighborhood |
|
||||
|
||||
### Residual-noise descriptors
|
||||
|
||||
print(features)
|
||||
| Variable | Modes | Description |
|
||||
| --- | --- | --- |
|
||||
| `residual_noise_energy` | Both | Mean squared median-filter residual |
|
||||
| `residual_noise_std` | Both | Grayscale residual standard deviation, or mean of the three channel standard deviations in RGB |
|
||||
| `residual_noise_std_global` | RGB only | Standard deviation over all RGB residual values jointly |
|
||||
| `residual_noise_mean_absolute` | Both | Mean absolute residual magnitude |
|
||||
|
||||
The median filter uses a `3x3` spatial neighborhood. In RGB mode its channel
|
||||
size is one, so color channels are never mixed.
|
||||
|
||||
## Explainable feature maps
|
||||
|
||||
Example output:
|
||||
| Variable | Grayscale shape | RGB shape | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| `fft_spectrum` | `(H, W)` | `(H, W)` | Normalized aggregate logarithmic spectrum |
|
||||
| `fft_spectrum_channels` | Not returned | `(H, W, 3)` | Independently normalized R, G, and B logarithmic spectra |
|
||||
| `brightness_map` | `(H, W)` | `(H, W, 3)` | Normalized image intensities |
|
||||
| `darkness_map` | `(H, W)` | `(H, W, 3)` | Intensity complements |
|
||||
| `gradient_map` | `(H, W)` | `(H, W)` | Normalized gradient; RGB channels are averaged before display normalization |
|
||||
| `local_entropy_map` | `(H, W)` | `(H, W)` | Normalized local entropy; RGB channels are averaged |
|
||||
| `local_variance_map` | `(H, W)` | `(H, W)` | Normalized local variance; RGB channels are averaged |
|
||||
| `residual_noise_raw` | `(H, W)` | `(H, W, 3)` | Signed median-filter residual |
|
||||
| `residual_noise` | `(H, W)` | `(H, W, 3)` | Square-root-enhanced residual visualization |
|
||||
|
||||
Visual normalization does not alter the quantitative descriptors.
|
||||
|
||||
## Output summary
|
||||
|
||||
{
|
||||
| Mode | Quantitative descriptors | Visual maps |
|
||||
| --- | ---: | ---: |
|
||||
| Grayscale | 14 | 8 |
|
||||
| RGB | 15 | 9 |
|
||||
|
||||
  "filename": "image.jpg",
|
||||
These counts exclude the four metadata fields.
|
||||
|
||||
  "spectral\_entropy": 2.31,
|
||||
## Interactive visualizations
|
||||
|
||||
  "anisotropy": 0.021,
|
||||
The plotting functions accept dictionaries, pandas `DataFrame` objects, and
|
||||
other inputs supported by `pandas.DataFrame`.
|
||||
|
||||
  "dominant\_frequency": 12.7,
|
||||
```python
|
||||
from spectralforensics import box_plot, line_plot
|
||||
|
||||
  "energy\_density": 58210.3,
|
||||
data = {"sample": [1, 2, 3, 4, 5],"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],}
|
||||
|
||||
  "high\_frequency\_ratio": 0.000091,
|
||||
line_plot(data, x="sample", y="spectral_entropy")
|
||||
box_plot(data, x="sample", y="anisotropy")
|
||||
```
|
||||
|
||||
  "mean\_energy": 58210.3
|
||||

|
||||

|
||||
|
||||
}
|
||||
Available plotting functions:
|
||||
|
||||
Extract features from an entire folder
|
||||
```python
|
||||
from spectralforensics import (bar_plot,box_plot,correlation_plot,line_plot,scatter_plot,)
|
||||
```
|
||||
|
||||
from spectralforensics import extract\_features\_from\_folder
|
||||
### Correlation analysis
|
||||
|
||||
```python
|
||||
from spectralforensics import correlation_plot
|
||||
correlation_data = {"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],"anisotropy": [0.10, 0.30, 0.20, 0.40, 0.25],"target": [0, 1, 0, 1, 1],}
|
||||
correlation_plot(correlation_data,correlation_columns=["spectral_entropy", "anisotropy", "target"],method="pearson",target="target",kind="barh",)
|
||||
```
|
||||
|
||||

|
||||
|
||||
df = extract\_features\_from\_folder(
|
||||
Set `show=False` to return the Matplotlib figure, axes, and correlation tables
|
||||
without opening the interactive window.
|
||||
|
||||
  folder\_path="dataset",
|
||||
## Interactive chart controls
|
||||
|
||||
  output\_csv="results.csv"
|
||||
The chart interface supports:
|
||||
|
||||
)
|
||||
- X-axis and multiple Y-variable selection
|
||||
- Category filtering
|
||||
- Zooming and panning
|
||||
- Navigation through previous views
|
||||
- Resetting the current view
|
||||
- Copying or saving a plot
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
print(df.head())
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
python examples/line_plot.py
|
||||
python examples/bar_plot.py
|
||||
python examples/scatter_plot.py
|
||||
python examples/box_plot.py
|
||||
python examples/correlation_plot.py
|
||||
```
|
||||
|
||||
## Project structure
|
||||
|
||||
Example output:
|
||||
```text
|
||||
spectralforensics/
|
||||
|-- examples/ # Plotting examples
|
||||
|-- img/ # README screenshots
|
||||
|-- src/
|
||||
| `-- spectralforensics/
|
||||
| |-- dataset.py # File and folder processing
|
||||
| |-- features.py # RGB feature extraction
|
||||
| |-- features_grayscale.py # Grayscale feature extraction
|
||||
| |-- plotting*.py # Interactive visualizations
|
||||
| `-- assets/ # Interface assets
|
||||
|-- pyproject.toml # Package metadata and dependencies
|
||||
`-- README.md
|
||||
```
|
||||
|
||||
## Research scope
|
||||
|
||||
SpectralForensics is a research and experimentation toolkit. Its descriptors can support statistical analysis and machine-learning models, but they should not be treated as standalone proof that an image is authentic or generated.
|
||||
|
||||
filename spectral\_entropy anisotropy dominant\_frequency energy\_density high\_frequency\_ratio
|
||||
## Authors
|
||||
|
||||
real\_001.jpg 1.21 0.012 11.0 91327 0.000037
|
||||
- Jose Garcia University of Leiria and Oeste (ULO)
|
||||
- Patricio Rodrigues University of Leiria and Oeste (ULO)
|
||||
- Carlos Grilo University of Leiria and Oeste (ULO)
|
||||
- Rolando Miragaia University of Leiria and Oeste (ULO)
|
||||
|
||||
dalle\_001.png 2.50 0.034 16.0 68150 0.000177
|
||||
## Contributing
|
||||
|
||||
flux\_001.jpg 2.51 0.019 6.0 61004 0.000064
|
||||
Contribuições?
|
||||
|
||||
Spectral Features
|
||||
|
||||
Spectral Entropy
|
||||
|
||||
|
||||
|
||||
Measures the randomness of the spectral energy distribution.
|
||||
|
||||
|
||||
|
||||
Higher values generally indicate a more dispersed frequency distribution.
|
||||
|
||||
|
||||
|
||||
Anisotropy
|
||||
|
||||
|
||||
|
||||
Measures directional imbalance of spectral energy.
|
||||
|
||||
|
||||
|
||||
Can reveal artificial directional patterns introduced by generative models.
|
||||
|
||||
|
||||
|
||||
Dominant Frequency
|
||||
|
||||
|
||||
|
||||
Represents the strongest non-DC frequency component in the Fourier spectrum.
|
||||
|
||||
|
||||
|
||||
Energy Density
|
||||
|
||||
|
||||
|
||||
Measures the amount of spectral energy normalized by image size.
|
||||
|
||||
|
||||
|
||||
High Frequency Ratio
|
||||
|
||||
|
||||
|
||||
Measures the proportion of energy located in high-frequency regions.
|
||||
|
||||
|
||||
|
||||
Useful for detecting residual artefacts and synthetic textures.
|
||||
|
||||
|
||||
|
||||
Scientific Background
|
||||
|
||||
|
||||
|
||||
Diffusion-based generative models such as:
|
||||
|
||||
|
||||
|
||||
Stable Diffusion
|
||||
|
||||
FLUX
|
||||
|
||||
DALL·E
|
||||
|
||||
Midjourney
|
||||
|
||||
|
||||
|
||||
generate images through iterative denoising processes.
|
||||
|
||||
|
||||
|
||||
Although visually realistic, these models may introduce latent frequency-domain artefacts that remain detectable through Fourier analysis.
|
||||
|
||||
|
||||
|
||||
SpectralForensics aims to provide an explainable framework for investigating these spectral fingerprints and supporting synthetic media detection research.
|
||||
|
||||
|
||||
|
||||
Research Applications
|
||||
|
||||
|
||||
|
||||
Potential applications include:
|
||||
|
||||
|
||||
|
||||
Digital Image Forensics
|
||||
|
||||
Deepfake Detection
|
||||
|
||||
AI-Generated Content Verification
|
||||
|
||||
Synthetic Media Analysis
|
||||
|
||||
Explainable AI (XAI)
|
||||
|
||||
Frequency-Domain Machine Learning
|
||||
|
||||
Cybersecurity Research
|
||||
|
||||
Roadmap
|
||||
|
||||
|
||||
|
||||
Future planned features:
|
||||
|
||||
|
||||
|
||||
Radial Distribution Analysis
|
||||
|
||||
Spectral Heatmaps
|
||||
|
||||
Average FFT Fingerprints
|
||||
|
||||
Statistical Validation (t-test, ANOVA, Mann–Whitney)
|
||||
|
||||
Random Forest Classification
|
||||
|
||||
SVM Classification
|
||||
|
||||
XGBoost Classification
|
||||
|
||||
Diffusion Model Attribution
|
||||
|
||||
Batch Processing Pipelines
|
||||
|
||||
S
|
||||
## Citation
|
||||
|
||||
Artigo?
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
from spectralforensics import bar_plot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
bar_plot(
|
||||
{"model": ["A", "B", "A", "B"],
|
||||
"spectral_entropy": [2.1, 2.5, 2.2, 2.9],
|
||||
"anisotropy": [0.1, -0.3, 0.2, 0.4]},
|
||||
x="model", y="spectral_entropy",
|
||||
title="Spectral analysis — bar plot",
|
||||
)
|
||||
@@ -0,0 +1,8 @@
|
||||
from spectralforensics import box_plot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
dataset = {"income": ["<=50K", "<=50K", "<=50K", ">50K", ">50K", ">50K"],
|
||||
"hours-per-week": [30, 40, 45, 40, 50, 60]}
|
||||
box_plot(dataset, x="income", y="hours-per-week",
|
||||
fill=False, gap=.1, palette="pastel")
|
||||
@@ -0,0 +1,12 @@
|
||||
from spectralforensics import correlation_plot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = correlation_plot(
|
||||
{"entropy": [2.1, 2.5, 2.2, 2.9, 2.6],
|
||||
"anisotropy": [0.1, 0.3, 0.2, 0.4, 0.25],
|
||||
"target": [0, 1, 0, 1, 1]},
|
||||
correlation_columns=["entropy", "anisotropy", "target"],
|
||||
method="pearson", cmap="coolwarm", center=0, annot=True, fmt=".2f",
|
||||
target="target", kind="barh", by="target",
|
||||
)
|
||||
@@ -0,0 +1,10 @@
|
||||
from spectralforensics import line_plot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
line_plot(
|
||||
{"sample": [1, 2, 3, 4, 5],
|
||||
"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6],
|
||||
"anisotropy": [0.1, 0.3, 0.2, 0.4, 0.25]},
|
||||
x="sample", y="spectral_entropy",
|
||||
)
|
||||
@@ -0,0 +1,11 @@
|
||||
from spectralforensics import scatter_plot
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
scatter_plot(
|
||||
{"dominant_frequency": [1, 2, 2, 3, 4],
|
||||
"anisotropy": [0.1, 0.3, 0.2, 0.4, 0.25],
|
||||
"spectral_entropy": [2.1, 2.5, 2.2, 2.9, 2.6]},
|
||||
x="dominant_frequency", y="anisotropy",
|
||||
title="Spectral analysis — scatter plot",
|
||||
)
|
||||
|
After Width: | Height: | Size: 131 KiB |
|
After Width: | Height: | Size: 357 KiB |
|
After Width: | Height: | Size: 25 KiB |
|
After Width: | Height: | Size: 134 KiB |
|
After Width: | Height: | Size: 5.8 KiB |
@@ -1,34 +1,42 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=61.0"]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "spectralforensics"
|
||||
version = "0.1.0"
|
||||
description = "Spectral feature extraction for image forensic analysis using FFT"
|
||||
description = "Explainable spectral forensic analysis"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
|
||||
authors = [
|
||||
{name = "Jose Garcia"}
|
||||
{ name = "José Garcia" },
|
||||
{ name = "Patricio Rodrigues" },
|
||||
{ name = "Carlos Grilo" },
|
||||
{ name = "Rolando Miragaia" }
|
||||
]
|
||||
|
||||
maintainers = [
|
||||
{ name = "José Garcia", email = "2023144112@my.ipleiria.pt" }
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"numpy",
|
||||
"pillow",
|
||||
"pandas",
|
||||
"pillow"
|
||||
"scipy",
|
||||
"matplotlib",
|
||||
"seaborn>=0.13",
|
||||
"scikit-image",
|
||||
"tqdm"
|
||||
]
|
||||
|
||||
keywords = [
|
||||
"fft",
|
||||
"image forensics",
|
||||
"deepfake detection",
|
||||
"frequency analysis",
|
||||
"spectral fingerprints"
|
||||
]
|
||||
[tool.setuptools]
|
||||
package-dir = {"" = "src"}
|
||||
|
||||
classifiers = [
|
||||
"Programming Language :: Python :: 3",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Operating System :: OS Independent"
|
||||
]
|
||||
[tool.setuptools.package-data]
|
||||
spectralforensics = ["assets/*.png"]
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
include = ["spectralforensics*"]
|
||||
|
||||
@@ -1,14 +1,186 @@
|
||||
Metadata-Version: 2.4
|
||||
Name: spectralforensics
|
||||
Version: 0.1.0
|
||||
Summary: Spectral feature extraction for image forensic analysis using FFT
|
||||
Author: Jose Garcia
|
||||
Keywords: fft,image forensics,deepfake detection,frequency analysis,spectral fingerprints
|
||||
Classifier: Programming Language :: Python :: 3
|
||||
Classifier: License :: OSI Approved :: MIT License
|
||||
Classifier: Operating System :: OS Independent
|
||||
Summary: Explainable spectral forensic analysis
|
||||
Author: José Garcia, Patricio Rodrigues, Carlos Grilo, Rolando Miragaia
|
||||
Maintainer-email: José Garcia <2023144112@my.ipleiria.pt>
|
||||
Requires-Python: >=3.10
|
||||
Description-Content-Type: text/markdown
|
||||
Requires-Dist: numpy
|
||||
Requires-Dist: pandas
|
||||
Requires-Dist: pillow
|
||||
Requires-Dist: pandas
|
||||
Requires-Dist: scipy
|
||||
Requires-Dist: matplotlib
|
||||
|
||||
\# SpectralForensics
|
||||
|
||||
|
||||
|
||||
SpectralForensics is an open-source Python library for image forensic analysis using \*\*Fast Fourier Transform (FFT)\*\* and frequency-domain feature extraction.
|
||||
|
||||
|
||||
|
||||
The library was designed to support research in digital image forensics, synthetic media detection, diffusion model analysis, and explainable artificial intelligence (XAI).
|
||||
|
||||
|
||||
|
||||
\## Features
|
||||
|
||||
|
||||
|
||||
SpectralForensics extracts a set of interpretable spectral descriptors from images:
|
||||
|
||||
|
||||
|
||||
\- Spectral Entropy
|
||||
|
||||
\- Anisotropy
|
||||
|
||||
\- Dominant Frequency
|
||||
|
||||
\- Energy Density
|
||||
|
||||
\- High Frequency Ratio
|
||||
|
||||
\- Mean Spectral Energy
|
||||
|
||||
|
||||
|
||||
These descriptors can be used for:
|
||||
|
||||
|
||||
|
||||
\- AI-generated image detection
|
||||
|
||||
\- Deepfake forensics
|
||||
|
||||
\- Diffusion model fingerprinting
|
||||
|
||||
\- Frequency-domain analysis
|
||||
|
||||
\- Explainable machine learning pipelines
|
||||
|
||||
|
||||
|
||||
\## Installation
|
||||
|
||||
|
||||
|
||||
\### Local installation
|
||||
|
||||
|
||||
|
||||
```bash
|
||||
|
||||
py -m pip install -e .
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
\\### Future PyPI installation
|
||||
|
||||
|
||||
|
||||
pip install spectralforensics
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
\\## Quick Start
|
||||
|
||||
|
||||
|
||||
\\### Extract Features from a Single Image
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
from spectralforensics import extract\\\_spectral\\\_features
|
||||
|
||||
|
||||
|
||||
features = extract\\\_spectral\\\_features("image.jpg")
|
||||
|
||||
|
||||
|
||||
print(features)
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\### Example Output
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
{
|
||||
|
||||
\  "filename": "image.jpg",
|
||||
|
||||
\  "spectral\\\_entropy": 2.31,
|
||||
|
||||
\  "anisotropy": 0.021,
|
||||
|
||||
\  "dominant\\\_frequency": 12.7,
|
||||
|
||||
\  "energy\\\_density": 58210.3,
|
||||
|
||||
\  "high\\\_frequency\\\_ratio": 0.000091,
|
||||
|
||||
\  "mean\\\_energy": 58210.3
|
||||
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\---
|
||||
|
||||
|
||||
|
||||
\### Extract Features from an Entire Folder
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
from spectralforensics import extract\\\_features\\\_from\\\_folder
|
||||
|
||||
|
||||
|
||||
df = extract\\\_features\\\_from\\\_folder(
|
||||
|
||||
\  folder\\\_path="dataset",
|
||||
|
||||
\  output\\\_csv="results.csv"
|
||||
|
||||
)
|
||||
|
||||
|
||||
|
||||
print(df.head())
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\### Example Output
|
||||
|
||||
|
||||
|
||||
| filename | spectral\_entropy | anisotropy | dominant\_frequency | energy\_density | high\_frequency\_ratio |
|
||||
|
||||
|-----------|----------------|------------|-------------------|----------------|---------------------|
|
||||
|
||||
| real\_001.jpg | 1.21 | 0.012 | 11.0 | 91327 | 0.000037 |
|
||||
|
||||
| dalle\_001.png | 2.50 | 0.034 | 16.0 | 68150 | 0.000177 |
|
||||
|
||||
| flux\_001.jpg | 2.51 | 0.019 | 6.0 | 61004 | 0.000064 |
|
||||
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
README.md
|
||||
pyproject.toml
|
||||
spectralforensics/__init__.py
|
||||
spectralforensics/dataset.py
|
||||
spectralforensics/features.py
|
||||
spectralforensics.egg-info/PKG-INFO
|
||||
spectralforensics.egg-info/SOURCES.txt
|
||||
spectralforensics.egg-info/dependency_links.txt
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
numpy
|
||||
pandas
|
||||
pillow
|
||||
pandas
|
||||
scipy
|
||||
matplotlib
|
||||
|
||||
@@ -1 +1 @@
|
||||
spectralforensics
|
||||
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
from .features import extract_spectral_features
|
||||
from .dataset import extract_features_from_folder
|
||||
|
||||
__version__ = "0.1.0"
|
||||
@@ -1,32 +0,0 @@
|
||||
from pathlib import Path
|
||||
import pandas as pd
|
||||
from .features import extract_spectral_features
|
||||
SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".bmp", ".tif", ".tiff", ".webp"}
|
||||
|
||||
def extract_features_from_folder(folder_path,output_csv=None,image_size=(512, 512),high_freq_threshold=0.65):
|
||||
folder_path = Path(folder_path)
|
||||
if not folder_path.exists():
|
||||
raise FileNotFoundError(f"Folder not found: {folder_path}")
|
||||
image_files = [
|
||||
file for file in folder_path.iterdir()
|
||||
if file.suffix.lower() in SUPPORTED_EXTENSIONS
|
||||
]
|
||||
results = []
|
||||
for image_file in image_files:
|
||||
try:
|
||||
features = extract_spectral_features(
|
||||
image_file,
|
||||
image_size=image_size,
|
||||
high_freq_threshold=high_freq_threshold
|
||||
)
|
||||
results.append(features)
|
||||
except Exception as e:
|
||||
results.append({
|
||||
"filename": image_file.name,
|
||||
"path": str(image_file),
|
||||
"error": str(e)
|
||||
})
|
||||
df = pd.DataFrame(results)
|
||||
if output_csv:
|
||||
df.to_csv(output_csv, index=False, encoding="utf-8-sig")
|
||||
return df
|
||||
@@ -1,44 +0,0 @@
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def extract_spectral_features(image_path,image_size=(512, 512), high_freq_threshold=0.65, center_exclusion=5):
|
||||
image_path = Path(image_path)
|
||||
img = Image.open(image_path).convert("L")
|
||||
img = img.resize(image_size)
|
||||
img_array = np.array(img, dtype=np.float32) / 255.0
|
||||
fft = np.fft.fft2(img_array)
|
||||
fft_shift = np.fft.fftshift(fft)
|
||||
magnitude = np.abs(fft_shift)
|
||||
energy = magnitude ** 2
|
||||
total_energy = np.sum(energy) + 1e-12
|
||||
mean_energy = np.mean(energy)
|
||||
probability = energy / total_energy
|
||||
spectral_entropy = -np.sum(probability * np.log2(probability + 1e-12))
|
||||
h, w = energy.shape
|
||||
cy, cx = h // 2, w // 2
|
||||
y, x = np.indices((h, w))
|
||||
radius = np.sqrt((x - cx) ** 2 + (y - cy) ** 2)
|
||||
horizontal_energy = np.sum(energy[cy - 2:cy + 3, :])
|
||||
vertical_energy = np.sum(energy[:, cx - 2:cx + 3])
|
||||
anisotropy = abs(horizontal_energy - vertical_energy) / (horizontal_energy + vertical_energy + 1e-12)
|
||||
energy_no_center = energy.copy()
|
||||
energy_no_center[cy - center_exclusion:cy + center_exclusion + 1,cx - center_exclusion:cx + center_exclusion + 1] = 0
|
||||
dominant_position = np.unravel_index(np.argmax(energy_no_center),energy_no_center.shape)
|
||||
dominant_frequency = radius[dominant_position]
|
||||
energy_density = total_energy / (h * w)
|
||||
max_radius = np.max(radius)
|
||||
high_freq_mask = radius > (high_freq_threshold * max_radius)
|
||||
high_frequency_energy = np.sum(energy[high_freq_mask])
|
||||
high_frequency_ratio = high_frequency_energy / total_energy
|
||||
return {
|
||||
"filename": image_path.name,
|
||||
"path": str(image_path),
|
||||
"spectral_entropy": float(spectral_entropy),
|
||||
"anisotropy": float(anisotropy),
|
||||
"dominant_frequency": float(dominant_frequency),
|
||||
"energy_density": float(energy_density),
|
||||
"high_frequency_ratio": float(high_frequency_ratio),
|
||||
"mean_energy": float(mean_energy)
|
||||
}
|
||||
@@ -0,0 +1,187 @@
|
||||
Metadata-Version: 2.4
|
||||
Name: spectralforensics
|
||||
Version: 0.1.0
|
||||
Summary: Explainable spectral forensic analysis
|
||||
Author: José Garcia, Patricio Rodrigues, Carlos Grilo, Rolando Miragaia
|
||||
Maintainer-email: José Garcia <2023144112@my.ipleiria.pt>
|
||||
Requires-Python: >=3.10
|
||||
Description-Content-Type: text/markdown
|
||||
Requires-Dist: numpy
|
||||
Requires-Dist: pillow
|
||||
Requires-Dist: pandas
|
||||
Requires-Dist: scipy
|
||||
Requires-Dist: matplotlib
|
||||
Requires-Dist: scikit-image
|
||||
|
||||
\# SpectralForensics
|
||||
|
||||
|
||||
|
||||
SpectralForensics is an open-source Python library for image forensic analysis using \*\*Fast Fourier Transform (FFT)\*\* and frequency-domain feature extraction.
|
||||
|
||||
|
||||
|
||||
The library was designed to support research in digital image forensics, synthetic media detection, diffusion model analysis, and explainable artificial intelligence (XAI).
|
||||
|
||||
|
||||
|
||||
\## Features
|
||||
|
||||
|
||||
|
||||
SpectralForensics extracts a set of interpretable spectral descriptors from images:
|
||||
|
||||
|
||||
|
||||
\- Spectral Entropy
|
||||
|
||||
\- Anisotropy
|
||||
|
||||
\- Dominant Frequency
|
||||
|
||||
\- Energy Density
|
||||
|
||||
\- High Frequency Ratio
|
||||
|
||||
\- Mean Spectral Energy
|
||||
|
||||
|
||||
|
||||
These descriptors can be used for:
|
||||
|
||||
|
||||
|
||||
\- AI-generated image detection
|
||||
|
||||
\- Deepfake forensics
|
||||
|
||||
\- Diffusion model fingerprinting
|
||||
|
||||
\- Frequency-domain analysis
|
||||
|
||||
\- Explainable machine learning pipelines
|
||||
|
||||
|
||||
|
||||
\## Installation
|
||||
|
||||
|
||||
|
||||
\### Local installation
|
||||
|
||||
|
||||
|
||||
```bash
|
||||
|
||||
py -m pip install -e .
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
\\### Future PyPI installation
|
||||
|
||||
|
||||
|
||||
pip install spectralforensics
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
\\## Quick Start
|
||||
|
||||
|
||||
|
||||
\\### Extract Features from a Single Image
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
from spectralforensics import extract\\\_spectral\\\_features
|
||||
|
||||
|
||||
|
||||
features = extract\\\_spectral\\\_features("image.jpg")
|
||||
|
||||
|
||||
|
||||
print(features)
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\### Example Output
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
{
|
||||
|
||||
\  "filename": "image.jpg",
|
||||
|
||||
\  "spectral\\\_entropy": 2.31,
|
||||
|
||||
\  "anisotropy": 0.021,
|
||||
|
||||
\  "dominant\\\_frequency": 12.7,
|
||||
|
||||
\  "energy\\\_density": 58210.3,
|
||||
|
||||
\  "high\\\_frequency\\\_ratio": 0.000091,
|
||||
|
||||
\  "mean\\\_energy": 58210.3
|
||||
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\---
|
||||
|
||||
|
||||
|
||||
\### Extract Features from an Entire Folder
|
||||
|
||||
|
||||
|
||||
```python
|
||||
|
||||
from spectralforensics import extract\\\_features\\\_from\\\_folder
|
||||
|
||||
|
||||
|
||||
df = extract\\\_features\\\_from\\\_folder(
|
||||
|
||||
\  folder\\\_path="dataset",
|
||||
|
||||
\  output\\\_csv="results.csv"
|
||||
|
||||
)
|
||||
|
||||
|
||||
|
||||
print(df.head())
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
\### Example Output
|
||||
|
||||
|
||||
|
||||
| filename | spectral\_entropy | anisotropy | dominant\_frequency | energy\_density | high\_frequency\_ratio |
|
||||
|
||||
|-----------|----------------|------------|-------------------|----------------|---------------------|
|
||||
|
||||
| real\_001.jpg | 1.21 | 0.012 | 11.0 | 91327 | 0.000037 |
|
||||
|
||||
| dalle\_001.png | 2.50 | 0.034 | 16.0 | 68150 | 0.000177 |
|
||||
|
||||
| flux\_001.jpg | 2.51 | 0.019 | 6.0 | 61004 | 0.000064 |
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
README.md
|
||||
pyproject.toml
|
||||
spectralforensics.egg-info/PKG-INFO
|
||||
spectralforensics.egg-info/SOURCES.txt
|
||||
spectralforensics.egg-info/dependency_links.txt
|
||||
spectralforensics.egg-info/requires.txt
|
||||
spectralforensics.egg-info/top_level.txt
|
||||
src/spectralforensics/__init__.py
|
||||
src/spectralforensics/dataset.py
|
||||
src/spectralforensics/features.py
|
||||
src/spectralforensics.egg-info/PKG-INFO
|
||||
src/spectralforensics.egg-info/SOURCES.txt
|
||||
src/spectralforensics.egg-info/dependency_links.txt
|
||||
src/spectralforensics.egg-info/requires.txt
|
||||
src/spectralforensics.egg-info/top_level.txt
|
||||
@@ -0,0 +1 @@
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
numpy
|
||||
pillow
|
||||
pandas
|
||||
scipy
|
||||
matplotlib
|
||||
scikit-image
|
||||
@@ -0,0 +1 @@
|
||||
spectralforensics
|
||||
@@ -0,0 +1,25 @@
|
||||
from .dataset import (
|
||||
extract_features,
|
||||
rename_generated_images,
|
||||
rename_real_images,
|
||||
)
|
||||
from .features import extract_features as extract_rgb_features
|
||||
from .features_grayscale import extract_grayscale_features
|
||||
from .plotting import line_plot
|
||||
from .plotting_bars import bar_plot
|
||||
from .plotting_scatter import scatter_plot
|
||||
from .plotting_correlation import correlation_plot
|
||||
from .plotting_box import box_plot
|
||||
|
||||
__all__ = [
|
||||
"extract_features",
|
||||
"extract_rgb_features",
|
||||
"extract_grayscale_features",
|
||||
"rename_real_images",
|
||||
"rename_generated_images",
|
||||
"line_plot",
|
||||
"bar_plot",
|
||||
"scatter_plot",
|
||||
"correlation_plot",
|
||||
"box_plot",
|
||||
]
|
||||
@@ -0,0 +1,128 @@
|
||||
import tkinter as tk
|
||||
from tkinter import ttk
|
||||
|
||||
class CheckCombobox(ttk.Combobox):
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
self._popup = None
|
||||
self._list = None
|
||||
self._index = -1
|
||||
super().__init__(*args, **kwargs)
|
||||
self.bind("<Button-1>", self._toggle_popup)
|
||||
self.bind("<Down>", self._open)
|
||||
self.bind("<space>", self._open)
|
||||
self.bind("<Escape>", self._close)
|
||||
self.bind("<Destroy>", self._destroyed, add="+")
|
||||
|
||||
def current(self, newindex=None):
|
||||
if newindex is not None:
|
||||
self._index = newindex
|
||||
return super().current(newindex)
|
||||
return self._index
|
||||
|
||||
def configure(self, cnf=None, **kwargs):
|
||||
result = super().configure(cnf, **kwargs)
|
||||
if self._popup is not None and ("values" in kwargs or
|
||||
isinstance(cnf, dict) and "values" in cnf):
|
||||
position = self._list.yview()[0]
|
||||
self._list.delete(0, "end")
|
||||
for value in self["values"]:
|
||||
self._list.insert("end", value)
|
||||
if 0 <= self._index < self._list.size():
|
||||
self._list.selection_set(self._index)
|
||||
self._list.activate(self._index)
|
||||
self._list.yview_moveto(position)
|
||||
return result
|
||||
|
||||
config = configure
|
||||
|
||||
def _toggle_popup(self, event=None):
|
||||
if self._popup is None:
|
||||
return self._open(event)
|
||||
return self._close(event)
|
||||
|
||||
def _open(self, event=None):
|
||||
if "disabled" in self.state() or self._popup is not None:
|
||||
return "break"
|
||||
values = self["values"]
|
||||
if not values:
|
||||
return "break"
|
||||
popup = self._popup = tk.Toplevel(self)
|
||||
popup.withdraw()
|
||||
popup.overrideredirect(True)
|
||||
popup.transient(self.winfo_toplevel())
|
||||
container = ttk.Frame(popup, borderwidth=1, relief="solid")
|
||||
container.pack(fill="both", expand=True)
|
||||
choices = self._list = tk.Listbox(container, height=min(12, len(values)),
|
||||
exportselection=False, activestyle="dotbox")
|
||||
scrollbar = ttk.Scrollbar(container, orient="vertical", command=choices.yview)
|
||||
choices.configure(yscrollcommand=scrollbar.set)
|
||||
scrollbar.pack(side="right", fill="y")
|
||||
choices.pack(side="left", fill="both", expand=True)
|
||||
for value in values:
|
||||
choices.insert("end", value)
|
||||
popup.update_idletasks()
|
||||
width = max(self.winfo_width(), popup.winfo_reqwidth())
|
||||
height = popup.winfo_reqheight()
|
||||
x = max(0, min(self.winfo_rootx(), self.winfo_screenwidth() - width))
|
||||
y = self.winfo_rooty() + self.winfo_height()
|
||||
if y + height > self.winfo_screenheight():
|
||||
y = max(0, self.winfo_rooty() - height)
|
||||
popup.geometry(f"{width}x{height}+{x}+{y}")
|
||||
choices.bind("<Button-1>", self._click)
|
||||
choices.bind("<space>", self._choose)
|
||||
choices.bind("<Return>", self._choose)
|
||||
popup.bind("<Escape>", self._close)
|
||||
popup.bind("<Button-1>", self._outside)
|
||||
popup.bind("<FocusOut>", self._focus_out)
|
||||
popup.deiconify()
|
||||
popup.grab_set()
|
||||
choices.focus_set()
|
||||
return "break"
|
||||
|
||||
def _click(self, event):
|
||||
index = self._list.nearest(event.y)
|
||||
bounds = self._list.bbox(index)
|
||||
if bounds and bounds[1] <= event.y < bounds[1] + bounds[3]:
|
||||
self._choose(index=index)
|
||||
return "break"
|
||||
|
||||
def _choose(self, event=None, index=None):
|
||||
if self._list is None or not self._list.size():
|
||||
return "break"
|
||||
self._index = self._list.index("active") if index is None else index
|
||||
self._list.selection_clear(0, "end")
|
||||
self._list.selection_set(self._index)
|
||||
self._list.activate(self._index)
|
||||
self.event_generate("<<ComboboxSelected>>")
|
||||
return "break"
|
||||
|
||||
def _outside(self, event):
|
||||
popup = self._popup
|
||||
if popup is not None and not (popup.winfo_rootx() <= event.x_root <
|
||||
popup.winfo_rootx() + popup.winfo_width() and
|
||||
popup.winfo_rooty() <= event.y_root <
|
||||
popup.winfo_rooty() + popup.winfo_height()):
|
||||
self._close()
|
||||
|
||||
def _focus_out(self, event):
|
||||
self.after_idle(self._check_focus)
|
||||
|
||||
def _check_focus(self):
|
||||
if self._popup is not None:
|
||||
focus = self.focus_get()
|
||||
if focus is None or not str(focus).startswith(str(self._popup) + "."):
|
||||
self._close()
|
||||
|
||||
def _close(self, event=None):
|
||||
if self._popup is not None:
|
||||
popup, self._popup = self._popup, None
|
||||
self._list = None
|
||||
if popup.winfo_exists():
|
||||
popup.grab_release()
|
||||
popup.destroy()
|
||||
return "break"
|
||||
|
||||
def _destroyed(self, event):
|
||||
if event.widget is self:
|
||||
self._close()
|
||||
|
After Width: | Height: | Size: 4.0 KiB |
|
After Width: | Height: | Size: 7.1 KiB |
|
After Width: | Height: | Size: 12 KiB |
@@ -0,0 +1,68 @@
|
||||
from pathlib import Path
|
||||
from uuid import uuid4
|
||||
import pandas as pd
|
||||
from .features import extract_features as extract_rgb_image_features
|
||||
from .features_grayscale import extract_grayscale_features
|
||||
from tqdm import tqdm
|
||||
|
||||
support_ext = {".jpg", ".jpeg", ".png", ".bmp", ".tif", ".tiff", ".webp"}
|
||||
def _rename_images(folder_path, prefix):
|
||||
folder = Path(folder_path)
|
||||
if not folder.exists():
|
||||
raise FileNotFoundError(f"Folder not found: {folder}")
|
||||
if not folder.is_dir():
|
||||
raise NotADirectoryError(f"Expected a folder: {folder}")
|
||||
images = sorted((file for file in folder.iterdir()if file.is_file() and file.suffix.lower() in support_ext),
|
||||
key=lambda file: file.name.lower(),)
|
||||
staged_images = []
|
||||
for image in images:
|
||||
temporary_path = folder / f".spectralforensics-{uuid4().hex}{image.suffix}"
|
||||
image.rename(temporary_path)
|
||||
staged_images.append((temporary_path, image.suffix.lower()))
|
||||
renamed_images = []
|
||||
for index, (temporary_path, suffix) in enumerate(staged_images, start=1):
|
||||
new_path = folder / f"{prefix}{index}{suffix}"
|
||||
temporary_path.rename(new_path)
|
||||
renamed_images.append(new_path)
|
||||
print(f"{len(renamed_images)} images renamed with the '{prefix}' prefix.")
|
||||
return renamed_images
|
||||
|
||||
def rename_real_images(folder_path):
|
||||
return _rename_images(folder_path, "real")
|
||||
|
||||
def rename_generated_images(folder_path):
|
||||
return _rename_images(folder_path, "gen")
|
||||
|
||||
def extract_features(path,output_csv=None,image_size=None,high_freq_threshold=0.65,center_exclusion=5,
|
||||
output="all",show_progress=True, color_mode="rgb",):
|
||||
path = Path(path)
|
||||
color_mode = str(color_mode).lower()
|
||||
extractors = {"rgb": extract_rgb_image_features,"grayscale": extract_grayscale_features,}
|
||||
if color_mode not in extractors:
|
||||
raise ValueError("color_mode must be either 'rgb' or 'grayscale'")
|
||||
extractor = extractors[color_mode]
|
||||
if image_size is None:
|
||||
image_size = (1024, 1024) if color_mode == "rgb" else (255, 255)
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(f"Path not found: {path}")
|
||||
if path.is_file():
|
||||
if path.suffix.lower() not in support_ext:
|
||||
raise ValueError(f"Unsupported image format: {path.suffix}")
|
||||
return extractor(path,image_size=image_size,high_freq_threshold=high_freq_threshold,center_exclusion=center_exclusion,output=output,)
|
||||
image_files = [file for file in path.iterdir() if file.is_file() and file.suffix.lower() in support_ext]
|
||||
iterator = (tqdm(image_files,total=len(image_files),desc="Extracting features",unit="image") if show_progress else image_files)
|
||||
results = []
|
||||
for image_file in iterator:
|
||||
try:
|
||||
features = extractor(image_file,
|
||||
image_size=image_size,
|
||||
high_freq_threshold=high_freq_threshold,
|
||||
center_exclusion=center_exclusion,
|
||||
output=output,)
|
||||
results.append(features)
|
||||
except Exception as e:
|
||||
results.append({"filename": image_file.name, "path": str(image_file), "error": str(e)})
|
||||
df = pd.DataFrame(results)
|
||||
if output_csv:
|
||||
df.to_csv(output_csv, index=False, encoding="utf-8-sig")
|
||||
return df
|
||||
@@ -0,0 +1,199 @@
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from scipy.ndimage import median_filter, sobel, uniform_filter
|
||||
from skimage.filters.rank import entropy
|
||||
from skimage.morphology import disk
|
||||
|
||||
def extract_features(image_path,image_size=(1024, 1024),high_freq_threshold=0.65,center_exclusion=5,output="all"):
|
||||
image_path = Path(image_path)
|
||||
output = str(output).lower()
|
||||
valid_outputs = {"all", "spectral", "spatial","residual", "quantitative", "visual"}
|
||||
if output not in valid_outputs:
|
||||
raise ValueError("output must be one of: 'all', 'spectral', ""'spatial', 'residual', 'quantitative', 'visual'")
|
||||
if not image_path.exists():
|
||||
raise FileNotFoundError(f"Image not found: {image_path}")
|
||||
if not image_path.is_file():
|
||||
raise ValueError(f"Expected an image file: {image_path}")
|
||||
with Image.open(image_path) as image:
|
||||
image = image.convert("RGB")
|
||||
image = image.resize(image_size)
|
||||
img_array = np.asarray(image, dtype=np.float32) / np.float32(255.0)
|
||||
height, width, channels = img_array.shape
|
||||
filename_lower = image_path.name.lower()
|
||||
if "gen" in filename_lower:
|
||||
target = "0"
|
||||
elif "real" in filename_lower:
|
||||
target = "1"
|
||||
else:
|
||||
target = None
|
||||
results = {"filename": image_path.name,"path": str(image_path),"target": target,"color_mode": "rgb",}
|
||||
calculate_spectral = output in {"all", "spectral", "quantitative", "visual"}
|
||||
calculate_spatial = output in {"all", "spatial", "quantitative", "visual"}
|
||||
calculate_residual = output in {"all", "residual", "quantitative", "visual"}
|
||||
return_spectral_values = output in {"all", "spectral", "quantitative"}
|
||||
return_spatial_values = output in {"all", "spatial", "quantitative"}
|
||||
return_residual_values = output in {"all", "residual", "quantitative"}
|
||||
return_visual_maps = output in {"all", "visual"}
|
||||
eps = 1e-12
|
||||
if calculate_spectral:
|
||||
fft = np.fft.fft2(img_array, axes=(0, 1))
|
||||
fft_shift = np.fft.fftshift(fft, axes=(0, 1))
|
||||
magnitude = np.abs(fft_shift).astype(np.float32)
|
||||
energy = np.square(magnitude, dtype=np.float32)
|
||||
center_y = height // 2
|
||||
center_x = width // 2
|
||||
y_indices, x_indices = np.indices((height, width), dtype=np.float32)
|
||||
radius = np.sqrt((x_indices - center_x) ** 2 + (y_indices - center_y) ** 2).astype(np.float32)
|
||||
maximum_radius = np.max(radius)
|
||||
high_frequency_mask = (radius > high_freq_threshold * maximum_radius)
|
||||
if return_spectral_values:
|
||||
spectral_entropy_channels = []
|
||||
anisotropy_channels = []
|
||||
dominant_frequency_channels = []
|
||||
energy_density_channels = []
|
||||
high_frequency_ratio_channels = []
|
||||
mean_energy_channels = []
|
||||
for c in range(channels):
|
||||
energy_c = energy[:, :, c]
|
||||
total_energy_c = float(np.sum(energy_c, dtype=np.float64)) + eps
|
||||
probability_c = energy_c / total_energy_c
|
||||
spectral_entropy_c = -np.sum(probability_c * np.log2(probability_c + eps),dtype=np.float64)
|
||||
horizontal_energy_c = np.sum(energy_c[max(0, center_y - 2):
|
||||
min(height, center_y + 3), :],dtype=np.float64)
|
||||
vertical_energy_c = np.sum(energy_c[:,max(0, center_x - 2):
|
||||
min(width, center_x + 3)],dtype=np.float64)
|
||||
anisotropy_c = (abs(horizontal_energy_c - vertical_energy_c) / (horizontal_energy_c +vertical_energy_c + eps))
|
||||
energy_no_center = energy_c.copy()
|
||||
y_start = max(0, center_y - center_exclusion)
|
||||
y_end = min(height, center_y + center_exclusion + 1)
|
||||
x_start = max(0, center_x - center_exclusion)
|
||||
x_end = min(width, center_x + center_exclusion + 1)
|
||||
energy_no_center[y_start:y_end,x_start:x_end] = 0
|
||||
dominant_position = np.unravel_index(np.argmax(energy_no_center),energy_no_center.shape)
|
||||
dominant_frequency_c = float(radius[dominant_position])
|
||||
energy_density_c = (total_energy_c / (height * width))
|
||||
high_frequency_energy_c = np.sum(energy_c[high_frequency_mask],dtype=np.float64)
|
||||
high_frequency_ratio_c = (high_frequency_energy_c /total_energy_c)
|
||||
mean_energy_c = np.mean(energy_c, dtype=np.float64)
|
||||
spectral_entropy_channels.append(spectral_entropy_c)
|
||||
anisotropy_channels.append(anisotropy_c)
|
||||
dominant_frequency_channels.append(dominant_frequency_c)
|
||||
energy_density_channels.append(energy_density_c)
|
||||
high_frequency_ratio_channels.append(high_frequency_ratio_c)
|
||||
mean_energy_channels.append(mean_energy_c)
|
||||
results.update({"spectral_entropy": float(np.mean(spectral_entropy_channels)),
|
||||
"anisotropy": float(np.mean(anisotropy_channels)),
|
||||
"dominant_frequency": float(np.mean(dominant_frequency_channels)),
|
||||
"energy_density": float(np.mean(energy_density_channels)),
|
||||
"high_frequency_ratio": float(np.mean(high_frequency_ratio_channels)),
|
||||
"mean_energy": float(np.mean(mean_energy_channels))})
|
||||
if return_visual_maps:
|
||||
fft_spectrum_raw = np.log1p(magnitude)
|
||||
fft_spectrum_channels = np.empty((height, width, channels),dtype=np.uint8)
|
||||
for c in range(channels):
|
||||
spectrum_c = fft_spectrum_raw[:, :, c]
|
||||
spectrum_min = spectrum_c.min()
|
||||
spectrum_max = spectrum_c.max()
|
||||
fft_spectrum_channels[:, :, c] = (255.0 *(spectrum_c - spectrum_min) / (spectrum_max - spectrum_min + eps)).astype(np.uint8)
|
||||
aggregated_energy = np.mean(energy, axis=2, dtype=np.float64)
|
||||
aggregated_magnitude = np.sqrt(aggregated_energy)
|
||||
aggregated_spectrum_raw = np.log1p(aggregated_magnitude)
|
||||
spectrum_min = aggregated_spectrum_raw.min()
|
||||
spectrum_max = aggregated_spectrum_raw.max()
|
||||
fft_spectrum = (255.0 *(aggregated_spectrum_raw - spectrum_min) / (spectrum_max - spectrum_min + eps)).astype(np.uint8)
|
||||
results.update({"fft_spectrum_channels": fft_spectrum_channels,"fft_spectrum": fft_spectrum})
|
||||
del fft
|
||||
del fft_shift
|
||||
del magnitude
|
||||
del energy
|
||||
del radius
|
||||
del y_indices
|
||||
del x_indices
|
||||
|
||||
if calculate_spatial:
|
||||
gradient_x = np.empty_like(img_array)
|
||||
gradient_y = np.empty_like(img_array)
|
||||
for c in range(channels):
|
||||
gradient_x[:, :, c] = sobel(
|
||||
img_array[:, :, c], axis=1, mode="reflect"
|
||||
)
|
||||
gradient_y[:, :, c] = sobel(
|
||||
img_array[:, :, c], axis=0, mode="reflect"
|
||||
)
|
||||
gradient_raw = np.sqrt(gradient_x ** 2 + gradient_y ** 2).astype(np.float32)
|
||||
local_mean = uniform_filter(img_array,size=(7, 7, 1),mode="reflect").astype(np.float32)
|
||||
local_squared_mean = uniform_filter(np.square(img_array, dtype=np.float32),size=(7, 7, 1),mode="reflect").astype(np.float32)
|
||||
local_variance_raw = np.maximum(local_squared_mean - local_mean ** 2,0).astype(np.float32)
|
||||
image_uint8 = np.asarray(np.clip(np.rint(img_array * 255.0), 0, 255),dtype=np.uint8)
|
||||
local_entropy_raw = np.empty((height, width, channels),dtype=np.float32)
|
||||
entropy_footprint = disk(3)
|
||||
for c in range(channels):
|
||||
local_entropy_raw[:, :, c] = entropy(image_uint8[:, :, c],footprint=entropy_footprint).astype(np.float32)
|
||||
if return_spatial_values:
|
||||
results.update({"mean_brightness": float(np.mean(img_array, dtype=np.float64)),
|
||||
"mean_darkness": float(np.mean(1.0 - img_array, dtype=np.float64)),
|
||||
"mean_gradient": float(np.mean(gradient_raw, dtype=np.float64)),
|
||||
"mean_local_entropy": float(np.mean(local_entropy_raw, dtype=np.float64)),
|
||||
"mean_local_variance": float(np.mean(local_variance_raw, dtype=np.float64))})
|
||||
|
||||
if return_visual_maps:
|
||||
brightness_map = img_array.copy()
|
||||
darkness_map = (1.0 - img_array).astype(np.float32)
|
||||
gradient_aggregated = np.mean(gradient_raw, axis=2)
|
||||
entropy_aggregated = np.mean(local_entropy_raw, axis=2)
|
||||
variance_aggregated = np.mean(local_variance_raw, axis=2)
|
||||
gradient_minimum = gradient_aggregated.min()
|
||||
gradient_maximum = gradient_aggregated.max()
|
||||
gradient_map = ((gradient_aggregated - gradient_minimum) /(gradient_maximum - gradient_minimum + eps)).astype(np.float32)
|
||||
entropy_minimum = entropy_aggregated.min()
|
||||
entropy_maximum = entropy_aggregated.max()
|
||||
local_entropy_map = ((entropy_aggregated - entropy_minimum) /(entropy_maximum - entropy_minimum + eps)).astype(np.float32)
|
||||
variance_minimum = variance_aggregated.min()
|
||||
variance_maximum = variance_aggregated.max()
|
||||
local_variance_map = ((variance_aggregated - variance_minimum) /(variance_maximum - variance_minimum + eps)).astype(np.float32)
|
||||
results.update({
|
||||
"brightness_map": brightness_map,
|
||||
"darkness_map": darkness_map,
|
||||
"gradient_map": gradient_map,
|
||||
"local_entropy_map": local_entropy_map,
|
||||
"local_variance_map": local_variance_map })
|
||||
|
||||
del gradient_x
|
||||
del gradient_y
|
||||
del gradient_raw
|
||||
del local_mean
|
||||
del local_squared_mean
|
||||
del local_variance_raw
|
||||
del local_entropy_raw
|
||||
del image_uint8
|
||||
|
||||
if calculate_residual:
|
||||
|
||||
denoised_image = median_filter(img_array,size=(3, 3, 1),mode="reflect").astype(np.float32)
|
||||
residual_noise_raw = (img_array - denoised_image).astype(np.float32)
|
||||
if return_residual_values:
|
||||
residual_noise_energy = np.mean(
|
||||
np.square(residual_noise_raw,dtype=np.float32),dtype=np.float64)
|
||||
residual_std_channels = np.std(residual_noise_raw,axis=(0, 1),dtype=np.float64)
|
||||
residual_noise_std = np.mean(residual_std_channels)
|
||||
residual_noise_std_global = np.std(residual_noise_raw,dtype=np.float64)
|
||||
residual_noise_mean_absolute = np.mean(np.abs(residual_noise_raw),dtype=np.float64)
|
||||
results.update({"residual_noise_energy": float(residual_noise_energy),
|
||||
"residual_noise_std": float(residual_noise_std),
|
||||
"residual_noise_std_global": float(residual_noise_std_global),
|
||||
"residual_noise_mean_absolute": float(residual_noise_mean_absolute)})
|
||||
if return_visual_maps:
|
||||
residual_absolute = np.abs(residual_noise_raw).astype(np.float32)
|
||||
residual_noise_display = np.empty_like(residual_absolute)
|
||||
for c in range(channels):
|
||||
residual_c = residual_absolute[:, :, c]
|
||||
residual_minimum = residual_c.min()
|
||||
residual_maximum = residual_c.max()
|
||||
residual_noise_display[:, :, c] = np.sqrt((residual_c - residual_minimum) / (residual_maximum - residual_minimum + eps)).astype(np.float32)
|
||||
results.update({"residual_noise_raw": residual_noise_raw,"residual_noise": residual_noise_display})
|
||||
del residual_absolute
|
||||
del denoised_image
|
||||
if not return_visual_maps:
|
||||
del residual_noise_raw
|
||||
return results
|
||||
@@ -0,0 +1,132 @@
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from scipy.ndimage import median_filter, sobel, uniform_filter
|
||||
from skimage.filters.rank import entropy
|
||||
from skimage.morphology import disk
|
||||
|
||||
|
||||
def extract_grayscale_features(image_path,image_size=(255, 255),high_freq_threshold=0.65,center_exclusion=5,output="all",):
|
||||
image_path = Path(image_path)
|
||||
output = str(output).lower()
|
||||
valid_outputs = {"all","spectral","spatial","residual","quantitative","visual",}
|
||||
if output not in valid_outputs:
|
||||
raise ValueError("output must be one of: 'all', 'spectral', 'spatial', 'residual', 'quantitative', 'visual'")
|
||||
if not image_path.exists():
|
||||
raise FileNotFoundError(f"Image not found: {image_path}")
|
||||
if not image_path.is_file():
|
||||
raise ValueError(f"Expected an image file: {image_path}")
|
||||
with Image.open(image_path) as image:
|
||||
image = image.convert("L")
|
||||
image = image.resize(image_size)
|
||||
img_array = np.asarray(image, dtype=np.float32) / np.float32(255.0)
|
||||
filename_lower = image_path.name.lower()
|
||||
if "gen" in filename_lower:
|
||||
target = "0"
|
||||
elif "real" in filename_lower:
|
||||
target = "1"
|
||||
else:
|
||||
target = None
|
||||
results = {"filename": image_path.name,"path": str(image_path),"target": target,"color_mode": "grayscale",}
|
||||
calculate_spectral = output in {"all", "spectral", "quantitative", "visual"}
|
||||
calculate_spatial = output in {"all", "spatial", "quantitative", "visual"}
|
||||
calculate_residual = output in {"all", "residual", "quantitative", "visual"}
|
||||
return_spectral_values = output in {"all", "spectral", "quantitative"}
|
||||
return_spatial_values = output in {"all", "spatial", "quantitative"}
|
||||
return_residual_values = output in {"all", "residual", "quantitative"}
|
||||
return_visual_maps = output in {"all", "visual"}
|
||||
eps = 1e-12
|
||||
|
||||
if calculate_spectral:
|
||||
fft = np.fft.fft2(img_array)
|
||||
fft_shift = np.fft.fftshift(fft)
|
||||
magnitude = np.abs(fft_shift).astype(np.float32)
|
||||
energy = np.square(magnitude, dtype=np.float32)
|
||||
total_energy = float(np.sum(energy, dtype=np.float64)) + eps
|
||||
height, width = energy.shape
|
||||
center_y = height // 2
|
||||
center_x = width // 2
|
||||
y_indices, x_indices = np.indices((height, width), dtype=np.float32)
|
||||
radius = np.sqrt((x_indices - center_x) ** 2 + (y_indices - center_y) ** 2).astype(np.float32)
|
||||
if return_spectral_values:
|
||||
probability = energy / total_energy
|
||||
spectral_entropy = -np.sum(probability * np.log2(probability + eps), dtype=np.float64)
|
||||
horizontal_energy = np.sum(energy[max(0, center_y - 2) : min(height, center_y + 3), :],dtype=np.float64,)
|
||||
vertical_energy = np.sum(energy[:, max(0, center_x - 2) : min(width, center_x + 3)],dtype=np.float64,)
|
||||
anisotropy = abs(horizontal_energy - vertical_energy) / (horizontal_energy + vertical_energy + eps)
|
||||
energy_no_center = energy.copy()
|
||||
y_start = max(0, center_y - center_exclusion)
|
||||
y_end = min(height, center_y + center_exclusion + 1)
|
||||
x_start = max(0, center_x - center_exclusion)
|
||||
x_end = min(width, center_x + center_exclusion + 1)
|
||||
energy_no_center[y_start:y_end, x_start:x_end] = 0
|
||||
dominant_position = np.unravel_index(np.argmax(energy_no_center), energy_no_center.shape)
|
||||
dominant_frequency = radius[dominant_position]
|
||||
energy_density = total_energy / (height * width)
|
||||
maximum_radius = np.max(radius)
|
||||
high_frequency_mask = radius > high_freq_threshold * maximum_radius
|
||||
high_frequency_energy = np.sum(energy[high_frequency_mask], dtype=np.float64)
|
||||
high_frequency_ratio = high_frequency_energy / total_energy
|
||||
mean_energy = np.mean(energy, dtype=np.float64)
|
||||
results.update({"spectral_entropy": float(spectral_entropy),
|
||||
"anisotropy": float(anisotropy),
|
||||
"dominant_frequency": float(dominant_frequency),
|
||||
"energy_density": float(energy_density),
|
||||
"high_frequency_ratio": float(high_frequency_ratio),
|
||||
"mean_energy": float(mean_energy),})
|
||||
if return_visual_maps:
|
||||
fft_spectrum_raw = np.log1p(magnitude)
|
||||
fft_minimum = fft_spectrum_raw.min()
|
||||
fft_maximum = fft_spectrum_raw.max()
|
||||
fft_spectrum = (255.0* (fft_spectrum_raw - fft_minimum)/ (fft_maximum - fft_minimum + eps)).astype(np.uint8)
|
||||
results["fft_spectrum"] = fft_spectrum
|
||||
|
||||
if calculate_spatial:
|
||||
gradient_x = sobel(img_array, axis=1, mode="reflect").astype(np.float32)
|
||||
gradient_y = sobel(img_array, axis=0, mode="reflect").astype(np.float32)
|
||||
gradient_raw = np.sqrt(gradient_x**2 + gradient_y**2).astype(np.float32)
|
||||
local_mean = uniform_filter(img_array, size=7, mode="reflect").astype(np.float32)
|
||||
local_squared_mean = uniform_filter(np.square(img_array, dtype=np.float32), size=7, mode="reflect").astype(np.float32)
|
||||
local_variance_raw = np.maximum(local_squared_mean - local_mean**2, 0).astype(np.float32)
|
||||
image_uint8 = (img_array * 255).astype(np.uint8)
|
||||
local_entropy_raw = entropy(image_uint8, footprint=disk(3)).astype(np.float32)
|
||||
if return_spatial_values:
|
||||
results.update({"mean_brightness": float(np.mean(img_array, dtype=np.float64)),
|
||||
"mean_darkness": float(np.mean(1.0 - img_array, dtype=np.float64)),
|
||||
"mean_gradient": float(np.mean(gradient_raw, dtype=np.float64)),
|
||||
"mean_local_entropy": float(np.mean(local_entropy_raw, dtype=np.float64)),
|
||||
"mean_local_variance": float(np.mean(local_variance_raw, dtype=np.float64)),})
|
||||
if return_visual_maps:
|
||||
brightness_map = img_array.copy()
|
||||
darkness_map = (1.0 - img_array).astype(np.float32)
|
||||
gradient_minimum = gradient_raw.min()
|
||||
gradient_maximum = gradient_raw.max()
|
||||
gradient_map = ((gradient_raw - gradient_minimum)/ (gradient_maximum - gradient_minimum + eps)).astype(np.float32)
|
||||
entropy_minimum = local_entropy_raw.min()
|
||||
entropy_maximum = local_entropy_raw.max()
|
||||
local_entropy_map = ((local_entropy_raw - entropy_minimum)/ (entropy_maximum - entropy_minimum + eps)).astype(np.float32)
|
||||
variance_minimum = local_variance_raw.min()
|
||||
variance_maximum = local_variance_raw.max()
|
||||
local_variance_map = ((local_variance_raw - variance_minimum)/ (variance_maximum - variance_minimum + eps)).astype(np.float32)
|
||||
results.update({"brightness_map": brightness_map,
|
||||
"darkness_map": darkness_map,
|
||||
"gradient_map": gradient_map,
|
||||
"local_entropy_map": local_entropy_map,
|
||||
"local_variance_map": local_variance_map,})
|
||||
|
||||
if calculate_residual:
|
||||
denoised_image = median_filter(img_array, size=3, mode="reflect").astype(np.float32)
|
||||
residual_noise_raw = (img_array - denoised_image).astype(np.float32)
|
||||
if return_residual_values:
|
||||
results.update({"residual_noise_energy": float(np.mean(
|
||||
np.square(residual_noise_raw, dtype=np.float32),
|
||||
dtype=np.float64,)),
|
||||
"residual_noise_std": float(np.std(residual_noise_raw, dtype=np.float64)),
|
||||
"residual_noise_mean_absolute": float(np.mean(np.abs(residual_noise_raw), dtype=np.float64)),})
|
||||
if return_visual_maps:
|
||||
residual_absolute = np.abs(residual_noise_raw).astype(np.float32)
|
||||
residual_minimum = residual_absolute.min()
|
||||
residual_maximum = residual_absolute.max()
|
||||
residual_noise_display = np.sqrt((residual_absolute - residual_minimum)/ (residual_maximum - residual_minimum + eps)).astype(np.float32)
|
||||
results.update({"residual_noise_raw": residual_noise_raw,"residual_noise": residual_noise_display,})
|
||||
return results
|
||||
@@ -0,0 +1,454 @@
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import colorsys, ctypes, io, sys
|
||||
from pathlib import Path
|
||||
from PIL import Image, ImageTk, ImageGrab
|
||||
from ctypes import wintypes
|
||||
from tkinter import messagebox
|
||||
from ._widgets import CheckCombobox
|
||||
|
||||
def _clip_segment(a, b):
|
||||
delta = b - a
|
||||
start, end = 0., 1.
|
||||
for axis in range(2):
|
||||
if delta[axis] == 0:
|
||||
if not 0 <= a[axis] <= 1:
|
||||
return None
|
||||
else:
|
||||
lo, hi = sorted(((0 - a[axis]) / delta[axis],
|
||||
(1 - a[axis]) / delta[axis]))
|
||||
start, end = max(start, lo), min(end, hi)
|
||||
if start > end:
|
||||
return None
|
||||
return np.array([a + start * delta, a + end * delta])
|
||||
|
||||
def _copy_canvas(canvas):
|
||||
if sys.platform != "win32":
|
||||
raise RuntimeError("Image copying is only available on Windows.")
|
||||
canvas.update_idletasks()
|
||||
x, y = canvas.winfo_rootx(), canvas.winfo_rooty()
|
||||
image = ImageGrab.grab((x, y, x + canvas.winfo_width(),
|
||||
y + canvas.winfo_height()), all_screens=True)
|
||||
buffer = io.BytesIO()
|
||||
image.convert("RGB").save(buffer, "BMP")
|
||||
data = buffer.getvalue()[14:]
|
||||
kernel, user = ctypes.windll.kernel32, ctypes.windll.user32
|
||||
kernel.GlobalAlloc.argtypes = [wintypes.UINT, ctypes.c_size_t]
|
||||
kernel.GlobalAlloc.restype = wintypes.HGLOBAL
|
||||
kernel.GlobalLock.argtypes = [wintypes.HGLOBAL]
|
||||
kernel.GlobalLock.restype = ctypes.c_void_p
|
||||
kernel.GlobalUnlock.argtypes = [wintypes.HGLOBAL]
|
||||
kernel.GlobalFree.argtypes = [wintypes.HGLOBAL]
|
||||
user.OpenClipboard.argtypes = [wintypes.HWND]
|
||||
user.SetClipboardData.argtypes = [wintypes.UINT, wintypes.HANDLE]
|
||||
user.SetClipboardData.restype = wintypes.HANDLE
|
||||
handle = kernel.GlobalAlloc(0x0002, len(data))
|
||||
if not handle:
|
||||
raise RuntimeError("Could not allocate memory for the image.")
|
||||
try:
|
||||
pointer = kernel.GlobalLock(handle)
|
||||
if not pointer:
|
||||
raise RuntimeError("Could not prepare the image.")
|
||||
try:
|
||||
ctypes.memmove(pointer, data, len(data))
|
||||
finally:
|
||||
kernel.GlobalUnlock(handle)
|
||||
if not user.OpenClipboard(canvas.winfo_id()):
|
||||
raise RuntimeError("The clipboard is busy. Please try again.")
|
||||
try:
|
||||
if not user.EmptyClipboard() or not user.SetClipboardData(8, handle):
|
||||
raise RuntimeError("Could not copy the image.")
|
||||
handle = None
|
||||
finally:
|
||||
user.CloseClipboard()
|
||||
finally:
|
||||
if handle:
|
||||
kernel.GlobalFree(handle)
|
||||
|
||||
def _category_color(index):
|
||||
rgb = colorsys.hsv_to_rgb((0.46 + index * 0.61803398875) % 1, .65, .95)
|
||||
return "#" + "".join(f"{round(channel * 255):02x}" for channel in rgb)
|
||||
|
||||
def _category_palette(count):
|
||||
"""Assign unique colors across the full set, independent of active filters."""
|
||||
if count > 256 ** 3:
|
||||
raise ValueError("Too many categories for distinct RGB colors.")
|
||||
palette, used = [], set()
|
||||
for index in range(count):
|
||||
rgb = colorsys.hsv_to_rgb((.46 + index / count) % 1, .65, .95)
|
||||
value = int("".join(f"{round(channel * 255):02x}" for channel in rgb), 16)
|
||||
# RGB rounding can merge colors when there are very many categories.
|
||||
while value in used:
|
||||
value = (value + 1) % (256 ** 3)
|
||||
used.add(value)
|
||||
palette.append(f"#{value:06x}")
|
||||
return palette
|
||||
|
||||
def _category_lines(values, labels, selected):
|
||||
lines = []
|
||||
ordered = values[np.argsort(values[:, 0], kind="stable")]
|
||||
for index in selected:
|
||||
start, end = np.searchsorted(ordered[:, 0], [index, index + 1])
|
||||
ys = ordered[start:end, 1]
|
||||
lines.append((labels[index], _category_color(index),
|
||||
np.column_stack((np.arange(1, len(ys) + 1), ys))))
|
||||
return lines
|
||||
|
||||
def _line_segments(points):
|
||||
finite = np.isfinite(points).all(axis=1)
|
||||
boundaries = np.flatnonzero(np.diff(np.r_[False, finite, False]))
|
||||
return [points[start:end] for start, end in boundaries.reshape(-1, 2)]
|
||||
|
||||
def _prepare_data(data, x, y):
|
||||
frame = pd.DataFrame(data).copy(deep=False)
|
||||
if not frame.columns.is_unique or not all(isinstance(c, str) for c in frame.columns):
|
||||
raise ValueError("Column names must be unique strings.")
|
||||
columns = [c for c in frame if pd.api.types.is_numeric_dtype(frame[c])
|
||||
and not pd.api.types.is_bool_dtype(frame[c])
|
||||
and not pd.api.types.is_complex_dtype(frame[c])]
|
||||
if not columns or frame.empty:
|
||||
raise ValueError("Provide non-empty data with at least one numeric column.")
|
||||
x = frame.columns[0] if x is None else x
|
||||
y = columns[min(1, len(columns) - 1)] if y is None else y
|
||||
if x not in frame.columns or y not in columns:
|
||||
raise ValueError("x must name a column in data; y must name a numeric column.")
|
||||
return frame, columns, x, y
|
||||
|
||||
def _plot_values(frame, x, y, sort_x=False):
|
||||
series = frame[x]
|
||||
numeric = (pd.api.types.is_numeric_dtype(series)
|
||||
and not pd.api.types.is_bool_dtype(series)
|
||||
and not pd.api.types.is_complex_dtype(series))
|
||||
labels = None
|
||||
if numeric:
|
||||
xs = series.to_numpy(dtype=float, na_value=np.nan)
|
||||
else:
|
||||
codes, categories = pd.factorize(series, sort=sort_x)
|
||||
xs = codes.astype(float)
|
||||
xs[codes < 0] = np.nan
|
||||
labels = [str(value) for value in categories]
|
||||
values = np.column_stack((xs, frame[y].to_numpy(dtype=float, na_value=np.nan)))
|
||||
if sort_x:
|
||||
values = values[np.argsort(values[:, 0], kind="stable")]
|
||||
return values, labels
|
||||
def line_plot(data, x=None, y=None, *, title="SpectralForensics", sort_x=True,
|
||||
master=None, block=None, max_categories=8):
|
||||
return _interactive_plot(data, x, y, title=title, sort_x=sort_x,
|
||||
master=master, block=block, max_categories=max_categories)
|
||||
|
||||
def _interactive_plot(data, x=None, y=None, *, title="SpectralForensics", sort_x=False,
|
||||
master=None, block=None, max_categories=8, kind="line",
|
||||
fill=False, gap=.1, palette="pastel"):
|
||||
if not isinstance(max_categories, int) or max_categories < 1:
|
||||
raise ValueError("max_categories must be a positive integer")
|
||||
if block is None:
|
||||
block = master is None
|
||||
if master is None:
|
||||
try:
|
||||
from IPython import get_ipython
|
||||
shell = get_ipython()
|
||||
except ImportError:
|
||||
shell = None
|
||||
if shell is not None and getattr(shell, "kernel", None) is not None:
|
||||
shell.enable_gui("tk")
|
||||
block = False
|
||||
frame, columns, x, y = _prepare_data(data, x, y)
|
||||
import tkinter as tk
|
||||
from tkinter import ttk
|
||||
window = tk.Tk() if master is None else tk.Toplevel(master)
|
||||
window.title(title)
|
||||
window.geometry("960x600")
|
||||
window.minsize(540, 360)
|
||||
controls = ttk.Frame(window, padding=12)
|
||||
controls.pack(fill="x")
|
||||
x_var = tk.StringVar(window, x)
|
||||
ttk.Label(controls, text="X").grid(row=0, column=0, padx=(0, 6))
|
||||
x_selector = ttk.Combobox(controls, textvariable=x_var, values=list(frame.columns),
|
||||
state="readonly", width=26)
|
||||
x_selector.grid(row=0, column=1, sticky="ew", padx=(0, 18))
|
||||
controls.columnconfigure(1, weight=1)
|
||||
controls.columnconfigure(3, weight=1)
|
||||
selected_variables = {y}
|
||||
quantitative_var = tk.StringVar(window, y)
|
||||
quantitative_filter = CheckCombobox(
|
||||
controls, textvariable=quantitative_var, state="readonly", width=28)
|
||||
ttk.Label(controls, text="Y" if kind == "box" else "Multiple Y").grid(
|
||||
row=0, column=2, padx=(0, 6), sticky="w")
|
||||
quantitative_filter.grid(row=0, column=3, sticky="ew")
|
||||
|
||||
def update_quantitative_choices():
|
||||
quantitative_filter.configure(values=[
|
||||
("☑ " if name in selected_variables else "☐ ") + name
|
||||
for name in columns])
|
||||
quantitative_var.set(", ".join(name for name in columns if name in selected_variables)
|
||||
or "Select Y variables")
|
||||
|
||||
update_quantitative_choices()
|
||||
if kind == "box":
|
||||
fill_var = tk.BooleanVar(window, fill)
|
||||
gap_var = tk.StringVar(window, str(gap))
|
||||
palette_var = tk.StringVar(window, palette)
|
||||
box_options = ttk.Frame(controls)
|
||||
box_options.grid(row=3, column=0, columnspan=4, sticky="w", pady=(8, 0))
|
||||
ttk.Checkbutton(box_options, text="Fill boxes", variable=fill_var,
|
||||
command=lambda: schedule()).pack(side="left", padx=(0, 12))
|
||||
ttk.Label(box_options, text="Gap").pack(side="left", padx=(0, 6))
|
||||
gap_selector = ttk.Combobox(box_options, textvariable=gap_var,
|
||||
values=[0, .05, .1, .2, .3, .4, .5, .75],
|
||||
state="readonly", width=6)
|
||||
gap_selector.pack(side="left", padx=(0, 12))
|
||||
gap_selector.bind("<<ComboboxSelected>>", lambda event: schedule())
|
||||
ttk.Label(box_options, text="Palette").pack(side="left", padx=(0, 6))
|
||||
palette_selector = ttk.Combobox(box_options, textvariable=palette_var,
|
||||
values=list(dict.fromkeys([palette, "pastel", "deep",
|
||||
"muted", "bright", "colorblind", "Set2", "husl"])),
|
||||
state="readonly", width=18)
|
||||
palette_selector.pack(side="left")
|
||||
palette_selector.bind("<<ComboboxSelected>>", lambda event: schedule())
|
||||
filter_label = ttk.Label(controls, text="Categories")
|
||||
status = ttk.Label(controls)
|
||||
category_var = tk.StringVar(window)
|
||||
category_selector = CheckCombobox(controls, textvariable=category_var,
|
||||
state="readonly", width=26)
|
||||
selected_categories = set()
|
||||
category_labels = []
|
||||
category_colors = []
|
||||
|
||||
def update_category_choices():
|
||||
all_selected = bool(category_labels) and len(selected_categories) == len(category_labels)
|
||||
category_selector.configure(values=[
|
||||
("☑ " if all_selected else "☐ ") + "Select all",
|
||||
*[("☑ " if index in selected_categories else "☐ ") + label
|
||||
for index, label in enumerate(category_labels)]])
|
||||
status.configure(text=f"{len(selected_categories)}/{len(category_labels)} categories selected; "
|
||||
+ ("all values shown as bars." if kind == "bar"
|
||||
else "distribution by category." if kind == "box"
|
||||
else "all observations shown as points." if kind == "scatter"
|
||||
else "all points shown per line."))
|
||||
category_var.set(", ".join(label for index, label in enumerate(category_labels)
|
||||
if index in selected_categories)
|
||||
or "Select categories")
|
||||
from matplotlib.figure import Figure
|
||||
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk
|
||||
from matplotlib.ticker import MaxNLocator, FixedLocator, FormatStrFormatter
|
||||
|
||||
plot_area = ttk.Frame(window)
|
||||
plot_area.pack(fill="both", expand=True)
|
||||
figure = Figure(figsize=(9.6, 6), facecolor="#101827", layout="constrained")
|
||||
axes = figure.subplots()
|
||||
canvas = FigureCanvasTkAgg(figure, master=plot_area)
|
||||
toolbar = NavigationToolbar2Tk(canvas, plot_area, pack_toolbar=False)
|
||||
toolbar.pack(side="bottom", fill="x")
|
||||
canvas.get_tk_widget().pack(fill="both", expand=True)
|
||||
filter_column = None
|
||||
cache_key, cache = None, None
|
||||
window.figure = figure
|
||||
window.axes = axes
|
||||
|
||||
def draw(event=None):
|
||||
nonlocal filter_column, cache_key, cache
|
||||
variables = [name for name in columns if name in selected_variables]
|
||||
variables = variables[:max_categories]
|
||||
key = (x_var.get(), tuple(variables))
|
||||
if key != cache_key:
|
||||
cache = {name: _plot_values(frame, key[0], name, sort_x)
|
||||
for name in variables}
|
||||
cache_key = key
|
||||
labels = (cache[variables[0]][1] if variables else
|
||||
_plot_values(frame, key[0], y, sort_x)[1])
|
||||
if kind == "box":
|
||||
_, box_categories = pd.factorize(frame[key[0]], sort=sort_x)
|
||||
labels = [str(value) for value in box_categories]
|
||||
if filter_column != x_var.get():
|
||||
filter_column = x_var.get()
|
||||
selected_categories.clear()
|
||||
category_labels[:] = labels if labels is not None else []
|
||||
category_colors[:] = _category_palette(len(category_labels))
|
||||
if labels is not None:
|
||||
selected_categories.update(range(len(labels)))
|
||||
update_category_choices()
|
||||
filter_label.grid(row=1, column=0, sticky="w", padx=(0, 6), pady=(8, 0))
|
||||
category_selector.grid(row=1, column=1, sticky="ew", padx=(0, 18), pady=(8, 0))
|
||||
status.grid(row=2, column=1, columnspan=3, sticky="w", pady=(4, 0))
|
||||
else:
|
||||
filter_label.grid_remove()
|
||||
category_selector.grid_remove()
|
||||
status.grid_remove()
|
||||
if labels is not None:
|
||||
selected = sorted(selected_categories)
|
||||
lines = []
|
||||
for name in variables:
|
||||
chosen = selected
|
||||
grouped = _category_lines(cache[name][0], labels, chosen)
|
||||
for index, line in zip(chosen, grouped):
|
||||
label = f"{line[0]} · {name}" if len(variables) > 1 else line[0]
|
||||
lines.append((label, category_colors[index], line[2]))
|
||||
else:
|
||||
lines = [(name, _category_color(columns.index(name)), cache[name][0])
|
||||
for name in variables]
|
||||
axes.clear()
|
||||
axes.set_facecolor("#101827")
|
||||
axes.tick_params(colors="#b9c7db")
|
||||
for spine in axes.spines.values():
|
||||
spine.set_color("#b9c7db")
|
||||
axes.set_title(title, color="#edf2fa")
|
||||
axes.set_xlabel("Observation by category — " + x_var.get()
|
||||
if labels is not None else x_var.get(), color="#edf2fa")
|
||||
axes.set_ylabel(variables[0] if len(variables) == 1 else "Quantitative variables",
|
||||
color="#edf2fa")
|
||||
axes.set_axisbelow(True)
|
||||
axes.grid(axis="y", color="#29364a")
|
||||
if kind == "box":
|
||||
from .plotting_box import _draw_boxplot
|
||||
_draw_boxplot(axes, frame, key[0], variables[0] if variables else None,
|
||||
box_categories, selected_categories, fill=fill_var.get(),
|
||||
gap=float(gap_var.get()), palette=palette_var.get())
|
||||
toolbar.update()
|
||||
canvas.draw_idle()
|
||||
return
|
||||
values = (np.concatenate([line[2] for line in lines]) if lines
|
||||
else np.empty((0, 2)))
|
||||
finite = values[np.isfinite(values).all(axis=1)]
|
||||
if not len(finite):
|
||||
axes.text(.5, .5, "No valid pairs in the selected categories",
|
||||
ha="center", va="center", transform=axes.transAxes, color="#edf2fa")
|
||||
else:
|
||||
group_width = None
|
||||
bar_handles = []
|
||||
if kind == "bar":
|
||||
xs = np.unique(finite[:, 0])
|
||||
group_width = .8 * (np.diff(xs).min() if len(xs) > 1 else 1)
|
||||
for index, (label, color, points) in enumerate(lines):
|
||||
if kind == "line":
|
||||
# NaNs keep the gaps between observations with missing data.
|
||||
clean = points.copy()
|
||||
clean[~np.isfinite(clean).all(axis=1)] = np.nan
|
||||
axes.plot(clean[:, 0], clean[:, 1], color=color, label=label,
|
||||
linewidth=1.5, marker="." if len(points) == 1 else None)
|
||||
else:
|
||||
points = points[np.isfinite(points).all(axis=1)]
|
||||
if kind == "scatter":
|
||||
axes.scatter(points[:, 0], points[:, 1], color=color, label=label, s=20)
|
||||
else:
|
||||
width = group_width / len(lines)
|
||||
offset = -group_width / 2 + (index + .5) * width
|
||||
from .plotting_bars import _draw_bars
|
||||
bar_handles.append(_draw_bars(
|
||||
axes, points[:, 0] + offset, points[:, 1], width * .9,
|
||||
color=color, label=label))
|
||||
if labels is not None:
|
||||
axes.xaxis.set_major_locator(MaxNLocator(integer=True))
|
||||
elif kind == "bar" and frame[key[0]].dropna().isin([0, 1]).all():
|
||||
axes.xaxis.set_major_locator(FixedLocator([0, 1]))
|
||||
axes.xaxis.set_major_formatter(FormatStrFormatter("%d"))
|
||||
axes.set_xlim(-.4, 1.4)
|
||||
if kind == "bar":
|
||||
# Avoid scanning every rectangle to find the "best" legend position.
|
||||
legend = axes.legend(handles=bar_handles, loc="upper right",
|
||||
facecolor="#101827", edgecolor="#29364a")
|
||||
else:
|
||||
legend = axes.legend(facecolor="#101827", edgecolor="#29364a")
|
||||
for text in legend.get_texts():
|
||||
text.set_color("#edf2fa")
|
||||
toolbar.update()
|
||||
canvas.draw_idle()
|
||||
|
||||
pending = None
|
||||
def render():
|
||||
nonlocal pending
|
||||
pending = None
|
||||
draw()
|
||||
|
||||
def schedule(event=None):
|
||||
nonlocal pending
|
||||
if pending is not None:
|
||||
window.after_cancel(pending)
|
||||
pending = window.after(120, render)
|
||||
|
||||
def close():
|
||||
if pending is not None:
|
||||
window.after_cancel(pending)
|
||||
figure.clear()
|
||||
window.destroy()
|
||||
|
||||
def reset_zoom():
|
||||
toolbar.home()
|
||||
|
||||
def scroll_zoom(event):
|
||||
if event.inaxes is not axes or event.xdata is None or event.ydata is None:
|
||||
return
|
||||
if not event.step:
|
||||
return
|
||||
toolbar.push_current()
|
||||
factor = 1 / 1.2 if event.step > 0 else 1.2
|
||||
for getter, setter, anchor in ((axes.get_xlim, axes.set_xlim, event.xdata),
|
||||
(axes.get_ylim, axes.set_ylim, event.ydata)):
|
||||
bounds = np.asarray(getter())
|
||||
setter(anchor + (bounds - anchor) * factor)
|
||||
toolbar.push_current()
|
||||
canvas.draw_idle()
|
||||
|
||||
canvas.mpl_connect("scroll_event", scroll_zoom)
|
||||
|
||||
def copy_plot():
|
||||
try:
|
||||
window.lift()
|
||||
canvas.draw()
|
||||
window.update_idletasks()
|
||||
_copy_canvas(canvas.get_tk_widget())
|
||||
except Exception as error:
|
||||
messagebox.showerror("Copy plot", str(error), parent=window)
|
||||
|
||||
actions = ttk.Frame(controls)
|
||||
actions.grid(row=1, column=3, sticky="ew", padx=(0, 18), pady=(8, 0))
|
||||
for filename, command in (("copy.png", copy_plot), ("undo.png", reset_zoom)):
|
||||
with Image.open(Path(__file__).parent / "assets" / filename) as icon:
|
||||
button_icon = ImageTk.PhotoImage(
|
||||
icon.convert("RGBA").resize((24, 24), Image.Resampling.LANCZOS),
|
||||
master=window)
|
||||
button = ttk.Button(actions, image=button_icon, command=command, padding=5)
|
||||
button.image = button_icon
|
||||
button.pack(side="left", padx=(0, 8))
|
||||
window.protocol("WM_DELETE_WINDOW", close)
|
||||
def select_quantitative(event=None):
|
||||
index = quantitative_filter.current()
|
||||
if index < 0:
|
||||
return
|
||||
name = columns[index]
|
||||
if kind == "box":
|
||||
selected_variables.clear()
|
||||
selected_variables.add(name)
|
||||
elif name in selected_variables:
|
||||
selected_variables.remove(name)
|
||||
elif len(selected_variables) < max_categories:
|
||||
selected_variables.add(name)
|
||||
else:
|
||||
window.bell()
|
||||
update_quantitative_choices()
|
||||
schedule()
|
||||
|
||||
x_selector.bind("<<ComboboxSelected>>", schedule)
|
||||
quantitative_filter.bind("<<ComboboxSelected>>", select_quantitative)
|
||||
def select_category(event=None):
|
||||
index = category_selector.current()
|
||||
if index < 0:
|
||||
return
|
||||
if index == 0:
|
||||
if len(selected_categories) == len(category_labels):
|
||||
selected_categories.clear()
|
||||
else:
|
||||
selected_categories.update(range(len(category_labels)))
|
||||
else:
|
||||
index -= 1
|
||||
if index in selected_categories:
|
||||
selected_categories.remove(index)
|
||||
else:
|
||||
selected_categories.add(index)
|
||||
update_category_choices()
|
||||
schedule()
|
||||
|
||||
category_selector.bind("<<ComboboxSelected>>", select_category)
|
||||
schedule()
|
||||
if block and master is None:
|
||||
window.mainloop()
|
||||
return window
|
||||
@@ -0,0 +1,27 @@
|
||||
from .plotting import _interactive_plot
|
||||
import numpy as np
|
||||
|
||||
__all__ = ["bar_plot"]
|
||||
|
||||
def _draw_bars(axes, x, y, width, *, color, label):
|
||||
"""Batch rectangles so large datasets do not create thousands of artists."""
|
||||
from matplotlib.collections import PolyCollection
|
||||
from matplotlib.patches import Patch
|
||||
|
||||
x, y = np.asarray(x, dtype=float), np.asarray(y, dtype=float)
|
||||
left, right = x - width / 2, x + width / 2
|
||||
vertices = np.empty((len(x), 4, 2), dtype=float)
|
||||
vertices[:, 0, 0] = vertices[:, 1, 0] = left
|
||||
vertices[:, 2, 0] = vertices[:, 3, 0] = right
|
||||
vertices[:, 0, 1] = vertices[:, 3, 1] = 0
|
||||
vertices[:, 1, 1] = vertices[:, 2, 1] = y
|
||||
collection = PolyCollection(vertices, facecolors=color, edgecolors="none")
|
||||
axes.add_collection(collection)
|
||||
axes.autoscale_view()
|
||||
return Patch(facecolor=color, label=label)
|
||||
|
||||
def bar_plot(data, x=None, y=None, *, title="SpectralForensics", sort_x=False,
|
||||
master=None, block=None, max_categories=8):
|
||||
return _interactive_plot(data, x, y, title=title, sort_x=sort_x,
|
||||
master=master, block=block,
|
||||
max_categories=max_categories, kind="bar")
|
||||
@@ -0,0 +1,43 @@
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import seaborn as sns
|
||||
|
||||
__all__ = ["box_plot"]
|
||||
|
||||
def _draw_boxplot(axes, frame, x, y, categories, selected, *, fill, gap, palette):
|
||||
axes.set_xlabel(x, color="#edf2fa")
|
||||
axes.set_ylabel(y or "Y", color="#edf2fa")
|
||||
chosen = sorted(selected)
|
||||
if y is None or not chosen:
|
||||
axes.text(.5, .5, "Select categories and a Y variable", ha="center",
|
||||
va="center", transform=axes.transAxes, color="#edf2fa")
|
||||
return
|
||||
codes = pd.Categorical(frame[x], categories=categories).codes
|
||||
values = frame[y].to_numpy(dtype=float, na_value=np.nan)
|
||||
valid = np.isin(codes, chosen) & np.isfinite(values)
|
||||
if not valid.any():
|
||||
axes.text(.5, .5, "No valid values in the selected categories",
|
||||
ha="center", va="center", transform=axes.transAxes, color="#edf2fa")
|
||||
return
|
||||
colors = sns.color_palette(palette, n_colors=len(categories))
|
||||
data = pd.DataFrame({"category": codes[valid], "value": values[valid]})
|
||||
sns.boxplot(data=data, x="category", y="value", hue="category",
|
||||
order=chosen, hue_order=chosen,
|
||||
palette={index: colors[index] for index in chosen},
|
||||
fill=fill, gap=gap, dodge=False, legend=False, ax=axes)
|
||||
axes.set_xticks(range(len(chosen)), labels=[str(categories[index]) for index in chosen])
|
||||
axes.set_xlabel(x, color="#edf2fa")
|
||||
axes.set_ylabel(y, color="#edf2fa")
|
||||
|
||||
def box_plot(data, x=None, y=None, *, title="SpectralForensics — Boxplot",
|
||||
fill=False, gap=.1, palette="pastel", sort_x=False,
|
||||
master=None, block=None):
|
||||
from .plotting import _interactive_plot
|
||||
if not 0 <= gap < 1:
|
||||
raise ValueError("gap must be between 0 (inclusive) and 1 (exclusive).")
|
||||
if not isinstance(palette, str):
|
||||
raise ValueError("palette must be a seaborn or Matplotlib palette name.")
|
||||
sns.color_palette(palette)
|
||||
return _interactive_plot(data, x, y, title=title, sort_x=sort_x,
|
||||
master=master, block=block, max_categories=1,
|
||||
kind="box", fill=fill, gap=gap, palette=palette)
|
||||
@@ -0,0 +1,298 @@
|
||||
import pandas as pd
|
||||
from ._widgets import CheckCombobox
|
||||
from matplotlib.figure import Figure
|
||||
import seaborn as sns
|
||||
import tkinter as tk
|
||||
from tkinter import ttk, messagebox
|
||||
from pathlib import Path
|
||||
from PIL import Image, ImageTk
|
||||
from matplotlib import colormaps
|
||||
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2Tk
|
||||
from .plotting import _copy_canvas
|
||||
__all__ = ["correlation_plot"]
|
||||
|
||||
def _correlation_figure(data, correlation_columns=None, *, method="pearson",
|
||||
cmap="coolwarm", center=0, annot=True, fmt=".2f",
|
||||
target=None, kind="barh", by=None, ascending=True,
|
||||
figsize=None, title="Feature Correlation Matrix",
|
||||
target_title=None, show=False, page="matrix"):
|
||||
frame = pd.DataFrame(data)
|
||||
if not frame.columns.is_unique:
|
||||
raise ValueError("Column names must be unique.")
|
||||
if kind not in ("bar", "barh"):
|
||||
raise ValueError("kind must be 'bar' or 'barh'.")
|
||||
if target is None and by is not None:
|
||||
raise ValueError("Provide target when using by.")
|
||||
if target is not None and by is not None and by != target:
|
||||
raise ValueError("by must name the target correlation column.")
|
||||
columns = (frame.select_dtypes(include=["number", "bool"]).columns.tolist()
|
||||
if correlation_columns is None else list(correlation_columns))
|
||||
if target is not None and target not in columns:
|
||||
columns.append(target)
|
||||
if not columns or len(set(columns)) != len(columns):
|
||||
raise ValueError("Provide a non-empty selection of unique numeric columns.")
|
||||
missing = [column for column in columns if column not in frame.columns]
|
||||
if missing:
|
||||
raise ValueError(f"Unknown columns: {missing}")
|
||||
invalid = [column for column in columns
|
||||
if not (pd.api.types.is_numeric_dtype(frame[column])
|
||||
or pd.api.types.is_bool_dtype(frame[column]))
|
||||
or pd.api.types.is_complex_dtype(frame[column])]
|
||||
if invalid:
|
||||
raise ValueError(f"Correlation requires numeric or boolean columns: {invalid}")
|
||||
if frame.empty:
|
||||
raise ValueError("Provide non-empty data.")
|
||||
if target is not None and len(columns) < 2:
|
||||
raise ValueError("Provide at least one feature besides the target.")
|
||||
corr_complete = frame[columns].corr(method=method)
|
||||
corr_target = None
|
||||
if target is not None:
|
||||
corr_target = (corr_complete[[target]].drop(index=target)
|
||||
.sort_values(by=target if by is None else by,
|
||||
ascending=ascending))
|
||||
if page not in ("matrix", "target"):
|
||||
raise ValueError("page must be 'matrix' or 'target'.")
|
||||
if page == "target" and target is None:
|
||||
raise ValueError("Select a target to display its bar plot.")
|
||||
figure = Figure(figsize=figsize or (14, 11))
|
||||
ax = figure.subplots()
|
||||
heatmap_ax = ax if page == "matrix" else None
|
||||
target_ax = ax if page == "target" else None
|
||||
if heatmap_ax is not None:
|
||||
sns.heatmap(corr_complete, annot=annot, fmt=fmt, cmap=cmap,
|
||||
center=center, vmin=-1, vmax=1, ax=heatmap_ax)
|
||||
heatmap_ax.set_title(title)
|
||||
if target_ax is not None:
|
||||
corr_target.plot(kind=kind, ax=target_ax, legend=False)
|
||||
method_name = method.capitalize() if isinstance(method, str) else "Custom"
|
||||
correlation_label = f"Correlation ({method_name}) with {target}"
|
||||
if kind == "barh":
|
||||
target_ax.set_xlabel(correlation_label)
|
||||
target_ax.set_ylabel("Feature")
|
||||
target_ax.axvline(0, linewidth=1, color="gray")
|
||||
target_ax.set_xlim(-1, 1)
|
||||
else:
|
||||
target_ax.set_ylabel(correlation_label)
|
||||
target_ax.set_xlabel("Feature")
|
||||
target_ax.axhline(0, linewidth=1, color="gray")
|
||||
target_ax.set_ylim(-1, 1)
|
||||
target_ax.set_title(target_title or f"Feature Correlation with {target}")
|
||||
figure.tight_layout()
|
||||
return {"figure": figure,
|
||||
"axes": {"heatmap": heatmap_ax, "target": target_ax},
|
||||
"corr_complete": corr_complete, "corr_target": corr_target}
|
||||
def correlation_plot(data, correlation_columns=None, *, method="pearson",
|
||||
cmap="coolwarm", center=0, annot=True, fmt=".2f",
|
||||
target=None, kind="barh", by=None, ascending=True,
|
||||
figsize=None, title="Feature Correlation Matrix",
|
||||
target_title=None, show=True, master=None, block=None):
|
||||
options = dict(method=method, cmap=cmap, center=center, annot=annot, fmt=fmt,
|
||||
target=target, kind=kind, by=by, ascending=ascending,
|
||||
figsize=figsize, title=title, target_title=target_title)
|
||||
# Validate and render before creating a window.
|
||||
result = _correlation_figure(data, correlation_columns, **options)
|
||||
if not show:
|
||||
return result
|
||||
|
||||
if block is None:
|
||||
block = master is None
|
||||
if master is None:
|
||||
try:
|
||||
from IPython import get_ipython
|
||||
shell = get_ipython()
|
||||
except ImportError:
|
||||
shell = None
|
||||
if shell is not None and getattr(shell, "kernel", None) is not None:
|
||||
shell.enable_gui("tk")
|
||||
block = False
|
||||
frame = pd.DataFrame(data)
|
||||
columns = [name for name in frame
|
||||
if (pd.api.types.is_numeric_dtype(frame[name])
|
||||
or pd.api.types.is_bool_dtype(frame[name]))
|
||||
and not pd.api.types.is_complex_dtype(frame[name])]
|
||||
selected = set(columns if correlation_columns is None else correlation_columns)
|
||||
window = tk.Tk() if master is None else tk.Toplevel(master)
|
||||
window.title(title)
|
||||
window.geometry("1200x800")
|
||||
window.minsize(800, 540)
|
||||
controls = ttk.Frame(window, padding=12)
|
||||
controls.pack(fill="x")
|
||||
status = tk.StringVar(window)
|
||||
feature_var = tk.StringVar(window)
|
||||
method_var = tk.StringVar(window, method if isinstance(method, str) else "Custom")
|
||||
cmap_var = tk.StringVar(window, cmap if isinstance(cmap, str) else cmap.name)
|
||||
center_var = tk.StringVar(window, "" if center is None else str(center))
|
||||
annot_var = tk.BooleanVar(window, annot)
|
||||
fmt_var = tk.StringVar(window, fmt)
|
||||
target_var = tk.StringVar(window, "(No target)" if target is None else target)
|
||||
kind_var = tk.StringVar(window, kind)
|
||||
by_var = tk.StringVar(window, "" if target is None else target)
|
||||
ascending_var = tk.BooleanVar(window, ascending)
|
||||
plot_area = ttk.Frame(window)
|
||||
plot_area.pack(fill="both", expand=True)
|
||||
window.result = result
|
||||
class CorrelationToolbar(NavigationToolbar2Tk):
|
||||
def back(self, *args):
|
||||
if window.result["axes"]["target"] is not None:
|
||||
navigate("matrix")
|
||||
|
||||
def forward(self, *args):
|
||||
if (window.result["axes"]["heatmap"] is not None
|
||||
and window.result["corr_target"] is not None):
|
||||
navigate("target")
|
||||
|
||||
def set_history_buttons(self):
|
||||
buttons = getattr(self, "_buttons", {})
|
||||
on_target = window.result["axes"]["target"] is not None
|
||||
if "Back" in buttons:
|
||||
buttons["Back"].configure(state="normal" if on_target else "disabled")
|
||||
if "Forward" in buttons:
|
||||
buttons["Forward"].configure(
|
||||
state="normal" if not on_target and window.result["corr_target"] is not None
|
||||
else "disabled")
|
||||
canvas = FigureCanvasTkAgg(result["figure"], master=plot_area)
|
||||
toolbar = CorrelationToolbar(canvas, plot_area, pack_toolbar=False)
|
||||
toolbar.pack(side="bottom", fill="x")
|
||||
canvas.get_tk_widget().pack(fill="both", expand=True)
|
||||
canvas.draw()
|
||||
window.result = result
|
||||
pending = None
|
||||
page = "matrix"
|
||||
page_label = tk.StringVar(window, "1/2 — Correlation matrix" if target is not None
|
||||
else "Correlation matrix")
|
||||
def update_plot():
|
||||
nonlocal pending, page
|
||||
pending = None
|
||||
try:
|
||||
chosen_target = None if target_var.get() == "(No target)" else target_var.get()
|
||||
current_page = page if chosen_target is not None else "matrix"
|
||||
current_method = method if method_var.get() == "Custom" else method_var.get()
|
||||
current = _correlation_figure(
|
||||
frame, [name for name in columns if name in selected],
|
||||
method=current_method, cmap=cmap_var.get(),
|
||||
center=float(center_var.get()) if center_var.get().strip() else None,
|
||||
annot=annot_var.get(), fmt=fmt_var.get(), target=chosen_target,
|
||||
kind=kind_var.get(), by=by_var.get() or None,
|
||||
ascending=ascending_var.get(), figsize=figsize, title=title,
|
||||
target_title=target_title, page=current_page)
|
||||
except Exception as error:
|
||||
status.set(str(error))
|
||||
return
|
||||
old_figure = canvas.figure
|
||||
canvas.figure = current["figure"]
|
||||
canvas.figure.set_canvas(canvas)
|
||||
canvas.figure.set_size_inches(old_figure.get_size_inches(), forward=False)
|
||||
toolbar.update()
|
||||
canvas.draw()
|
||||
old_figure.clear()
|
||||
window.result = current
|
||||
page = current_page
|
||||
toolbar.set_history_buttons()
|
||||
page_label.set(("1/2 — Correlation matrix" if page == "matrix"
|
||||
else "2/2 — Target correlation") if chosen_target is not None
|
||||
else "Correlation matrix")
|
||||
status.set("Updated.")
|
||||
def schedule(event=None):
|
||||
nonlocal pending
|
||||
if pending is not None:
|
||||
window.after_cancel(pending)
|
||||
pending = window.after(200, update_plot)
|
||||
def combo(label, variable, values, row, column, readonly=True, multiple=False):
|
||||
ttk.Label(controls, text=label).grid(row=row, column=column, sticky="w", padx=5, pady=4)
|
||||
widget_type = CheckCombobox if multiple else ttk.Combobox
|
||||
widget = widget_type(controls, textvariable=variable, values=values,
|
||||
state="readonly" if readonly else "normal", width=22)
|
||||
widget.grid(row=row, column=column + 1, sticky="ew", padx=5, pady=4)
|
||||
widget.bind("<<ComboboxSelected>>", schedule)
|
||||
widget.bind("<Return>", schedule)
|
||||
return widget
|
||||
features = combo("Features", feature_var, [], 0, 0, multiple=True)
|
||||
def refresh_features():
|
||||
features.configure(values=[
|
||||
("☑ " if len(selected) == len(columns) else "☐ ") + "Select all",
|
||||
*[("☑ " if name in selected else "☐ ") + str(name) for name in columns]])
|
||||
feature_var.set(f"{len(selected)}/{len(columns)} features selected")
|
||||
def select_feature(event=None):
|
||||
index = features.current()
|
||||
if index == 0:
|
||||
if len(selected) == len(columns):
|
||||
selected.clear()
|
||||
else:
|
||||
selected.update(columns)
|
||||
elif index > 0:
|
||||
name = columns[index - 1]
|
||||
if name in selected:
|
||||
selected.remove(name)
|
||||
else:
|
||||
selected.add(name)
|
||||
refresh_features()
|
||||
schedule()
|
||||
features.bind("<<ComboboxSelected>>", select_feature)
|
||||
refresh_features()
|
||||
combo("Method", method_var,
|
||||
["pearson", "spearman", "kendall"] + (["Custom"] if callable(method) else []), 0, 2)
|
||||
combo("Color map", cmap_var, sorted(colormaps), 0, 4)
|
||||
target_selector = combo("Target", target_var, ["(No target)", *columns], 1, 0)
|
||||
combo("Plot type", kind_var, ["bar", "barh"], 1, 2)
|
||||
by_selector = combo("Sort by", by_var, [] if target is None else [target], 1, 4)
|
||||
def change_target(event=None):
|
||||
value = "" if target_var.get() == "(No target)" else target_var.get()
|
||||
by_var.set(value)
|
||||
by_selector.configure(values=[value] if value else [])
|
||||
schedule()
|
||||
target_selector.bind("<<ComboboxSelected>>", change_target)
|
||||
for label, variable, column in (("Center (empty = automatic)", center_var, 0),
|
||||
("Format (fmt)", fmt_var, 2)):
|
||||
ttk.Label(controls, text=label).grid(row=2, column=column, sticky="w", padx=5)
|
||||
entry = ttk.Entry(controls, textvariable=variable, width=22)
|
||||
entry.grid(row=2, column=column + 1, sticky="ew", padx=5)
|
||||
entry.bind("<Return>", schedule)
|
||||
entry.bind("<FocusOut>", schedule)
|
||||
checks = ttk.Frame(controls)
|
||||
checks.grid(row=2, column=4, columnspan=2, sticky="w")
|
||||
ttk.Checkbutton(checks, text="Annotations", variable=annot_var, command=schedule).pack(side="left")
|
||||
ttk.Checkbutton(checks, text="Ascending order", variable=ascending_var, command=schedule).pack(side="left")
|
||||
def copy_plot():
|
||||
try:
|
||||
window.lift()
|
||||
canvas.draw()
|
||||
_copy_canvas(canvas.get_tk_widget())
|
||||
except Exception as error:
|
||||
messagebox.showerror("Copy plot", str(error), parent=window)
|
||||
actions = ttk.Frame(controls)
|
||||
actions.grid(row=3, column=0, columnspan=6, sticky="w", pady=(8, 0))
|
||||
def navigate(destination):
|
||||
nonlocal page, pending
|
||||
if pending is not None:
|
||||
window.after_cancel(pending)
|
||||
pending = None
|
||||
old_page = page
|
||||
page = destination
|
||||
update_plot()
|
||||
if window.result["axes"]["heatmap"] is not None and page == "target":
|
||||
page = old_page
|
||||
elif window.result["axes"]["target"] is not None and page == "matrix":
|
||||
page = old_page
|
||||
ttk.Label(actions, textvariable=page_label).pack(side="left", padx=5)
|
||||
for filename, command in (("refresh-page-option.png", update_plot),
|
||||
("copy.png", copy_plot), ("undo.png", toolbar.home)):
|
||||
with Image.open(Path(__file__).parent / "assets" / filename) as icon:
|
||||
button_icon = ImageTk.PhotoImage(
|
||||
icon.convert("RGBA").resize((24, 24), Image.Resampling.LANCZOS),
|
||||
master=window)
|
||||
button = ttk.Button(actions, image=button_icon, command=command, padding=5)
|
||||
button.image = button_icon
|
||||
button.pack(side="left", padx=5)
|
||||
ttk.Label(controls, textvariable=status, wraplength=1050).grid(
|
||||
row=4, column=0, columnspan=6, sticky="w", padx=5, pady=(5, 0))
|
||||
for column in (1, 3, 5):
|
||||
controls.columnconfigure(column, weight=1)
|
||||
def close():
|
||||
if pending is not None:
|
||||
window.after_cancel(pending)
|
||||
window.result["figure"].clear()
|
||||
window.destroy()
|
||||
window.protocol("WM_DELETE_WINDOW", close)
|
||||
if block and master is None:
|
||||
window.mainloop()
|
||||
return window
|
||||
@@ -0,0 +1,8 @@
|
||||
from .plotting import _interactive_plot
|
||||
__all__ = ["scatter_plot"]
|
||||
|
||||
def scatter_plot(data, x=None, y=None, *, title="SpectralForensics", sort_x=False,
|
||||
master=None, block=None, max_categories=8):
|
||||
return _interactive_plot(data, x, y, title=title, sort_x=sort_x,
|
||||
master=master, block=block,
|
||||
max_categories=max_categories, kind="scatter")
|
||||