Could you please provide the code for train and neuron selection ?

[CHR-ray]

we are replicating you results, but the trigger generation part always return high loss trigger pattern. Could you please provide the code for train and neuron selection ?

this is my implementation part of selecting neuron in torch:

if isinstance(net.getattr(layer_name), torch.nn.modules.Linear): #weight is (n,m)
connect_level = torch.abs(net.getattr(layer_name).weight).sum(1) # if is a matrix, then all rows is summed.
elif isinstance(net.getattr(layer_name), torch.nn.modules.Conv2d): #weight is (c_out, c_in, h, w)
connect_level = torch.abs(net.getattr(layer_name).weight).sum([1,2,3])

[ain-soph]

I think it’s in this file.

https://github.com/PurduePAML/TrojanNN/blob/master/code/get_neuron_idx_from_weight.py

[ain-soph]

Here's my understanding to fix your implementation. Hope it's correct.

1. you shall calculate the weight of the next layer, rather than the selected layer.
2. You are summing the wrong dimension. For the next layer, you shall sum over out_channelsorout_features, so that you get the tensor with same length as number of neurons from previous layer output.

[JohnnyQAQ]

Here's my understanding to fix your implementation. Hope it's correct.

1. you shall calculate the weight of the preceding layer, rather than the selected layer.
2. You are summing the wrong dimension. For the preceding layer, you shall sum over out_channels/out_features, so that you get the tensor with same length as number of neurons from previous layer output.

Hello, I would like to ask, since it is to check the parameters of the previous layer. But what is written in the code is ”To select neuron in fc6 we check the weight in layer fc7“, shouldn't fc7 be the next layer of fc6?

[ain-soph]

@JohnnyQAQ
Thanks for correcting my typo. Not preceding layer but the next layer. To select fc6 neurons, we need to calculate fc7 weights.

[JohnnyQAQ]

@JohnnyQAQ Thanks for correcting my typo. Not preceding layer but the next layer. To select fc6 neurons, we need to calculate fc7 weights.

Thank you very much for your answer. What confuses me is that in the "Internal Neuron Selection" part in the TrojanNN paper is indeed using "the preceding layer", but I checked the code of TrojanNN and TrojanZoo and found that the implementation of this part both seems to search for the parameter W of the next layer. I'm wondering if there is something wrong with my understanding, could you please explain this part? Is there something wrong with my understanding of "the preceding layer" in the paper?

[ain-soph]

@JohnnyQAQ I'm the author of TrojanZoo and its TrojanNN implementation is following the code of this repo.

I consulted the original author Yingqi Liu and I believe it is correct to use the next layer. (Although I also doubt whether this neuron activation story makes sense... The actual running doesn't show a crazy activation gap like it is in paper. )

Here is my personal explanation about neuron activation: They want to select the neurons which are well-connected to next layer, and later optimize their trigger to maximize those activations. Therefore, it should measure the weights matrix of next layer, and use mean/sum to remove the out_features dimension, so that we get a vector whose length equals to number of neurons (output dimension of previous layer).

[JohnnyQAQ]

@ain-soph Thank you very much for your explanation.