# End-to-end walkthrough
## Setup
Install the package, then run the sections below. Each step imports what
it needs from `tell_me_why` (or fastai for the dataset):
``` bash
pip install tmw-xai
```
## 1. Binary data: cats vs dogs (PETS)
Each image file name starts with the breed. fastai’s usual rule labels
**cats** when the breed name starts with an uppercase letter and
**dogs** otherwise.
Sample images from the Oxford-IIIT Pet dataset used in this walkthrough:
█
|----------------------------------------| 0.00% [0/811706944 00:00
## 2. Train an explainable model
We use the built-in **`AAE`** (ResNet-34 encoder + xAAEnet blocks). For
a **custom encoder**, use **`EncoderWithAAEBlocks`** instead (documented
under *Add xAAEnet Blocks to a User Encoder*).
Build fastai dataloaders on PETS (300×300 here; match
`AAE(input_size=…)`), then run `train_xaaenet` (three phases; increase
`epochs_*` for real runs).
### Training dataloaders
For `train_xaaenet`, we label **cats** vs **dogs** from the file name
(uppercase breed → cat) and resize to **300×300** (same value as
`AAE(input_size=300)` below).
``` python
from fastai.vision.all import (
CategoryBlock,
DataBlock,
ImageBlock,
RandomSplitter,
Resize,
)
def pet_species(path):
"""Cat breeds start with an uppercase letter in the PETS file names."""
return "cat" if path.name[0].isupper() else "dog"
# Subsample for a quicker first run (remove [:400] for the full dataset)
pet_items = list(get_image_files(pets_path))[:400]
dblock = DataBlock(
blocks=(ImageBlock, CategoryBlock),
get_items=lambda _: pet_items,
splitter=RandomSplitter(valid_pct=0.2, seed=42),
get_y=pet_species,
item_tfms=Resize(300),
)
dls = dblock.dataloaders(pets_path, bs=16, num_workers=0)
dls.vocab, len(dls.train_ds), len(dls.valid_ds)
```
Choose which species is **class A** (`target_score = 1.0`). With
`pet_species`, the vocabulary is `cat` and `dog`:
``` python
class_a_name = "cat"
class_b_name = "dog"
class_a_name, class_b_name, dls.vocab
```
``` python
from tell_me_why.model_aae import AAE
from tell_me_why.training import train_xaaenet
model = AAE(input_size=300, encoding_dims=128)
learn = train_xaaenet(
model,
dls,
epochs_adv=1,
epochs_ae=1,
epochs_classif=2,
save_tsne=False,
save_pls=False,
extract_latent=True,
show_latent_figures=False,
)
```
## 3. Validation latent codes `z` and binary targets
Collect one latent row per validation image by running the trained model
on the validation dataloader. Row order matches `learn.dls.valid` (no
shuffling on the validation set).
**Do not reorder** images before the feature table in the next step.
``` python
import numpy as np
import torch
@torch.no_grad()
def latent_and_targets_from_split(learn, ds_idx=1):
learn.model.eval()
z_parts, y_parts = [], []
for xb, yb in learn.dls[ds_idx]:
learn.model(xb)
z_parts.append(learn.model.z.detach().cpu())
y_parts.append(yb.cpu())
z = torch.cat(z_parts).numpy()
targets = torch.cat(y_parts).view(-1).numpy()
return z, targets
z_val, targets_val = latent_and_targets_from_split(learn, ds_idx=1)
target_score = (targets_val == learn.dls.vocab.o2i[class_a_name]).astype(np.float64)
z_val.shape, target_score.mean()
```
``` python
# Same row order as z_val / target_score
image_paths = [str(p) for p in learn.dls.valid.items]
len(image_paths), image_paths[0]
```
## 4. Feature score table
`compute_feature_score_table` turns each image path into numeric cues
(brightness, color, texture, …). Row order must match `z_val` on the
validation split.
The table below uses the same PETS sample paths as above. After
training, score the full validation set.
``` python
from pathlib import Path
from tell_me_why.feature_scores import compute_feature_score_table
def format_feature_table(df):
"""Drop full paths; show file names first (same layout as the Feature scores example)."""
out = df.copy()
out["image_name"] = out["Source_File_Path"].map(lambda path: Path(path).name)
out = out.drop(columns="Source_File_Path")
return out[["image_name", *[col for col in out.columns if col != "image_name"]]]
```
``` python
pets_scores = compute_feature_score_table(
pet_images,
score_names=[
"brightness",
"variance",
"redness_dominance",
"symmetry_error",
"fft_high_frequency_ratio",
],
on_error="raise",
)
format_feature_table(pets_scores).round(4)
```
Computing brightness: 0%| | 0/8 [00:00
|
image_name |
brightness |
variance |
redness_dominance |
symmetry_error |
fft_high_frequency_ratio |
| 0 |
shiba_inu_14.jpg |
0.7555 |
0.0176 |
0.5491 |
0.0889 |
0.5315 |
| 1 |
Bombay_58.jpg |
0.1711 |
0.0477 |
0.7016 |
0.1534 |
0.4465 |
| 2 |
Siamese_173.jpg |
0.4284 |
0.0479 |
0.5613 |
0.2105 |
0.5394 |
| 3 |
miniature_pinscher_4.jpg |
0.5688 |
0.0625 |
0.5165 |
0.2563 |
0.4899 |
| 4 |
beagle_31.jpg |
0.4799 |
0.0355 |
0.4072 |
0.2172 |
0.6502 |
| 5 |
chihuahua_75.jpg |
0.4795 |
0.0594 |
0.7384 |
0.2793 |
0.4448 |
| 6 |
Bengal_58.jpg |
0.6254 |
0.0477 |
0.5762 |
0.2379 |
0.4307 |
| 7 |
Russian_Blue_130.jpg |
0.5452 |
0.0665 |
0.5519 |
0.1784 |
0.4253 |
### Scores on the validation split
After training, build the table on **every** validation image (all
default feature columns). Use the same paths as in step 3, in the same
order as `z_val`.
``` python
df_features = compute_feature_score_table(image_paths)
format_feature_table(df_features).round(4)
```
## 5. Classification interpretation figures
Compare latent axis PLS1 to each feature. The section *Reading the
alignment panels* and *Reading the importance ranking* on the
**Classification Interpretation** page explains how to read the figures.
``` python
from tell_me_why.visualization import run_pls_feature_figures
out = run_pls_feature_figures(
z_val,
target_score,
df_features,
target_label=class_a_name,
mask_positive=target_score.astype(bool),
positive_label=class_a_name,
negative_label=class_b_name,
save_dir=learn.path / learn.model_dir / "interpretation",
show=False,
)
out["importance_rank"][:5]
```
## Reading the results (short checklist)
- **Importance ranking** — features at the top have the strongest signed
r² with PLS1 toward class A; near-zero bars were weakly aligned on
this axis.
- **Alignment panels** — diagonal green line + separated grey/blue
clouds suggest a pixel cue that co-varies with the latent decision
axis; a flat line suggests little linear link.
- **Causality** — alignment is a **comparison** between latent space and
hand-crafted scores, not a proof of what neurons implement.
## Next steps
- Train longer and on the full PETS split (or your own binary dataset).
- Swap `AAE` for `EncoderWithAAEBlocks` when you bring a custom encoder.
- Reuse the same `z` + `df_features` + `run_pls_feature_figures`
pipeline after any binary xAAEnet training run.