Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example based #166

Merged
merged 32 commits into from
Oct 8, 2024
Merged
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9c1f39e
gitignore: remove test output files
AntoninPoche Oct 7, 2024
ed7aac0
commons: add utils for example based
AntoninPoche Oct 7, 2024
1b74123
types: add dataset or tensor type for example based
AntoninPoche Oct 7, 2024
09ea4f4
example based: create functions to manipulate datasets
AntoninPoche Oct 7, 2024
c5af51c
example based: introduce the base method
AntoninPoche Oct 7, 2024
690278e
example based: introduce base projection
AntoninPoche Oct 7, 2024
1b73a82
example based: introduce base search method
AntoninPoche Oct 7, 2024
f381fc9
example based: add several projections
AntoninPoche Oct 7, 2024
f11b5f8
example based: add commons for search methods
AntoninPoche Oct 7, 2024
11c73f3
example based: add knn and filter knn search methods
lucashervier Oct 7, 2024
ba3409e
example based: add similar examples
AntoninPoche Oct 7, 2024
be9c35b
example based: add counterfactuals
lucashervier Oct 7, 2024
89b363e
example based: add semifactuals
lucashervier Oct 7, 2024
109fa26
example based: add prototypes
ChafikBak Oct 7, 2024
1278689
example based: add an image ploting function
AntoninPoche Oct 7, 2024
127300e
example based tests: add datasets operations tests
AntoninPoche Oct 7, 2024
2fb379c
example based tests: test knn and filter knn
lucashervier Oct 7, 2024
d43c53a
example based tests: test projections
AntoninPoche Oct 7, 2024
5e11075
example based tests: test similar examples and cole
AntoninPoche Oct 7, 2024
84dec31
example based tests: test counterfactuals and semifactuals
lucashervier Oct 7, 2024
c6e6ad1
example based tests: test prototypes
ChafikBak Oct 7, 2024
9a4a080
example based tests: test torch support
AntoninPoche Oct 7, 2024
33bc623
example based tests: test plotting function
AntoninPoche Oct 7, 2024
4cc1e06
example based tests: add test utils
AntoninPoche Oct 7, 2024
ac3f20b
example based docs: add documentation base
lucashervier Oct 7, 2024
287b38d
example based docs: add documentation for similar examples
lucashervier Oct 7, 2024
854b65a
example based docs: add documentation for counterfactuals
lucashervier Oct 7, 2024
e0c7ab8
example based docs: add documentation for semifactuals
lucashervier Oct 7, 2024
429ebed
example based docs: add documentation for prototypes
ChafikBak Oct 7, 2024
5bd707b
setup: add example based test for torch and ignore pylint too many po…
AntoninPoche Oct 7, 2024
09645e7
concepts tests: move random test to debug notebooks
AntoninPoche Oct 8, 2024
341b9b6
Bump version: 1.3.3 → 1.4.0
AntoninPoche Oct 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
example based docs: add documentation for similar examples
Co-authored-by: Antonin Poché <antonin.poche@irt-saintexupery.com>
lucashervier and AntoninPoche committed Oct 7, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit 287b38dc777014421cbcca310bc55c5f5b17e65e
78 changes: 78 additions & 0 deletions docs/api/example_based/similar_examples/cole.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# COLE: Contributions Oriented Local Explanations

<sub>
<img src="https://upload.wikimedia.org/wikipedia/commons/d/d0/Google_Colaboratory_SVG_Logo.svg" width="20">
</sub> [View colab tutorial](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
<sub>
<img src="https://upload.wikimedia.org/wikipedia/commons/9/91/Octicons-mark-github.svg" width="20">
</sub> [View source](https://github.com/deel-ai/xplique/blob/master/xplique/example_based/similar_examples.py) |
📰 [Paper](https://researchrepository.ucd.ie/handle/10197/11064)

COLE for Contributions Oriented Local Explanations was introduced by Kenny & Keane in 2019.

!!! quote
Our method COLE is based on the premise that the contributions of features in a model’s classification represent the most sensible basis to inform case-based explanations.

-- <cite>[COLE paper](https://researchrepository.ucd.ie/handle/10197/11064)</cite>[^1]

The core idea of the COLE approach is to use [attribution maps](../../../attributions/api_attributions/) to define a relevant search space for the K-Nearest Neighbors (KNN) search.

More specifically, the COLE approach is based on the following steps:

- (1) Given an input sample $x$, compute the attribution map $A(x)$

- (2) Consider the projection space defined by: $p: x \rightarrow A(x) \odot x$ ($\odot$ denotes the element-wise product)

- (3) Perform a KNN search in the projection space to find the most similar training samples

!!! info
In the original paper, the authors focused on Multi-Layer Perceptrons (MLP) and three attribution methods (Hadamard, LPR, Integrated Gradient, and DeepLift). We decided to implement a COLE method that generalizes to a more broader range of Neural Networks and attribution methods (see [API Attributions documentation](../../../attributions/api_attributions/) to see the list of methods available).

!!! tips
The original paper shown that the hadamard product between the latent space and the gradient was the best method. Hence we optimized the code for this method. Setting the `attribution_method` argument to `"gradient"` will run much faster.

## Example

```python
from xplique.example_based import Cole

# load the training dataset and the model
cases_dataset = ... # load the training dataset
model = ... # load the model

# load the test samples
test_samples = ... # load the test samples to search for

# parameters
k = 3
case_returns = "all" # elements returned by the explain function
distance = "euclidean"
attribution_method = "gradient",
latent_layer = "last_conv" # where to split your model for the projection

# instantiate the Cole object
cole = Cole(
cases_dataset=cases_dataset,
model=model,
k=k,
attribution_method=attribution_method,
latent_layer=latent_layer,
case_returns=case_returns,
distance=distance,
)

# search the most similar samples with the COLE method
similar_samples = cole.explain(
inputs=test_samples,
targets=None, # not necessary with default operator, they are computed internally
)
```

## Notebooks

- [**Example-based Methods**: Getting started](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF)

{{xplique.example_based.similar_examples.Cole}}

[^1]: [Twin-Systems to Explain Artificial Neural Networks using Case-Based Reasoning:
Comparative Tests of Feature-Weighting Methods in ANN-CBR Twins for XAI (2019)](https://researchrepository.ucd.ie/handle/10197/11064)
57 changes: 57 additions & 0 deletions docs/api/example_based/similar_examples/similar_examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Similar-Examples

<sub>
<img src="https://upload.wikimedia.org/wikipedia/commons/d/d0/Google_Colaboratory_SVG_Logo.svg" width="20">
</sub> [View colab tutorial](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF) |
<sub>
<img src="https://upload.wikimedia.org/wikipedia/commons/9/91/Octicons-mark-github.svg" width="20">
</sub> [View source](https://github.com/deel-ai/xplique/blob/master/xplique/example_based/similar_examples.py)

We designate here as *Similar Examples* all methods that given an input sample, search for the most similar **training** samples given a distance function `distance`. Furthermore, one can define the search space using a `projection` function (see [Projections](../../projections/)). This function should map an input sample to the search space where the distance function is defined and meaningful (**e.g.** the latent space of a Convolutional Neural Network).
Then, a K-Nearest Neighbors (KNN) search is performed to find the most similar samples in the search space.

## Example

```python
from xplique.example_based import SimilarExamples

cases_dataset = ... # load the training dataset
targets = ... # load the one-hot encoding of predicted labels of the training dataset

# parameters
k = 5
distance = "euclidean"
case_returns = ["examples", "nuns"]

# define the projection function
def custom_projection(inputs: tf.Tensor, np.ndarray, targets: tf.Tensor, np.ndarray = None):
'''
Example of projection,
inputs are the elements to project.
targets are optional parameters to orientate the projection.
'''
projected_inputs = # do some magic on inputs, it should use the model.
return projected_inputs

# instantiate the SimilarExamples object
sim_ex = SimilarExamples(
cases_dataset=cases_dataset,
targets_dataset=targets,
k=k,
projection=custom_projection,
distance=distance,
)

# load the test samples and targets
test_samples = ... # load the test samples to search for
test_targets = ... # load the one-hot encoding of the test samples' predictions

# search the most similar samples with the SimilarExamples method
similar_samples = sim_ex.explain(test_samples, test_targets)
```

# Notebooks

- [**Example-based Methods**: Getting started](https://colab.research.google.com/drive/1gA7mhWhWzdKholZWkTvAg4FzFnzS8NHF)

{{xplique.example_based.similar_examples.SimilarExamples}}