Issue with compute_causal_penalty

[she3r]

This is the implementation of computation of causal penalty i received in the draft notebook:

def compute_causal_penalty(samples, adjacency_matrix, sample_order, categorical=None):
    """
    Calculate the inconsistency of samples with the given adjacency matrix.

    Parameters:
    - adjacency_matrix: np.ndarray, the adjacency matrix from DirectLiNGAM
    - samples: np.ndarray, the samples to evaluate (num_samples x num_features)

    Returns:
    - inconsistency: float, a measure of how inconsistent the samples are with the adjacency matrix
    """
    num_samples, num_features = samples.shape
    inconsistency = 0.0

    if categorical is None:
        categorical = [False] * len(sample_order)
    # Iterate through each feature and its causal parents
    for i in sample_order:
        parents = np.where(adjacency_matrix[i, :] != 0)[0]
        if len(parents) > 0:
            # Predicted values based on parents
            predicted_values = np.dot(samples[:, parents], adjacency_matrix[i, parents])
            # Calculate the inconsistency as the mean squared error
            mse = (samples[:, i] - predicted_values) ** 2
            if categorical[i]:
                mse = min((1, mse))
            inconsistency += mse

    return np.sqrt(np.mean(inconsistency)) / len(sample_order)

1. I suggest different calculation for categorical features:
   if categorical[i]:
   mse = (samples[:, i].round() != predicted_values.round()).astype(float)
   else:
   mse = (samples[:, i] - predicted_values) ** 2

2. I think categorical features should be entirely skipped from calculating the causality or we need something other than DirectLiNGAM for this.

[sbobek]

The code below assured that the cost for categorical feature (assumed OHE) will never be larger than 1 (as of wrongly set OHE featre).
I'd suggest keeping this, if no better idea, as it allows to ad penalty to incorrectly set OHE features, which is fine in case o DirectLinGam.

if categorical[i]:
         mse = min((1, mse))

[she3r]

def black_box_function(trial):
    # Suggest values for each feature based on bounds
    kwargs = {}
    for i, key in enumerate(features_order):
        if categorical_indicator[i]:
            kwargs[key] = trial.suggest_categorical(key, list(range(int(bounds[key][0]), int(bounds[key][1]) + 1)))
        else:
            kwargs[key] = trial.suggest_float(key, bounds[key][0], bounds[key][1], log=False)

    kwargs = update_masked_features_dict(features_order, masked_features, **kwargs)
    # todo rounding
    cf = np.array([[int(round(kwargs[key])) if categorical_indicator[i] else kwargs[key]
                    for i, key in enumerate(features_order)]]).reshape(1, -1)

This is how counterfactual candidates from optuna are defined – which only indicates that we are not currently enforcing the selection of only one option in a feature broken down into cases, i.e., it can be feature_valueA = 0/1, feature_valueB = 0/1, feature_valueC = 0/1, all of which can be 1 or 0. If we definitely want to rework this, that's fine, but the current implementation + EBM were prepared for a scenario without one-hot encoding, and in particular, the current implementation is not ready for one-hot encoding in the dataset.
Additionally, if we want to rework, my original suggestion holds

if categorical[i]:
   mse = (samples[:, i].round() != predicted_values.round()).astype(float)

and additionally we enforce using OHE by some proper input data validation