captum/attr/_core/neuron/neuron_gradient_shap.py [13:97]: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - r""" Implements gradient SHAP for a neuron in a hidden layer based on the implementation from SHAP's primary author. For reference, please, view: https://github.com/slundberg/shap\ #deep-learning-example-with-gradientexplainer-tensorflowkeraspytorch-models A Unified Approach to Interpreting Model Predictions http://papers.nips.cc/paper\ 7062-a-unified-approach-to-interpreting-model-predictions GradientShap approximates SHAP values by computing the expectations of gradients by randomly sampling from the distribution of baselines/references. It adds white noise to each input sample `n_samples` times, selects a random baseline from baselines' distribution and a random point along the path between the baseline and the input, and computes the gradient of the neuron with index `neuron_selector` with respect to those selected random points. The final SHAP values represent the expected values of `gradients * (inputs - baselines)`. GradientShap makes an assumption that the input features are independent and that the explanation model is linear, meaning that the explanations are modeled through the additive composition of feature effects. Under those assumptions, SHAP value can be approximated as the expectation of gradients that are computed for randomly generated `n_samples` input samples after adding gaussian noise `n_samples` times to each input for different baselines/references. In some sense it can be viewed as an approximation of integrated gradients by computing the expectations of gradients for different baselines. Current implementation uses Smoothgrad from `NoiseTunnel` in order to randomly draw samples from the distribution of baselines, add noise to input samples and compute the expectation (smoothgrad). """ def __init__( self, forward_func: Callable, layer: Module, device_ids: Union[None, List[int]] = None, multiply_by_inputs: bool = True, ) -> None: r""" Args: forward_func (callable): The forward function of the model or any modification of it layer (torch.nn.Module): Layer for which neuron attributions are computed. The output size of the attribute method matches the dimensions of the inputs or ouputs of the neuron with index `neuron_selector` in this layer, depending on whether we attribute to the inputs or outputs of the neuron. Currently, it is assumed that the inputs or the outputs of the neurons in this layer, depending on which one is used for attribution, can only be a single tensor. device_ids (list(int)): Device ID list, necessary only if forward_func applies a DataParallel model. This allows reconstruction of intermediate outputs from batched results across devices. If forward_func is given as the DataParallel model itself, then it is not necessary to provide this argument. multiply_by_inputs (bool, optional): Indicates whether to factor model inputs' multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs' multiplier isn't factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104 In case of Neuron Gradient SHAP, if `multiply_by_inputs` is set to True, the sensitivity scores for scaled inputs are being multiplied by (inputs - baselines). """ NeuronAttribution.__init__(self, forward_func, layer, device_ids) GradientAttribution.__init__(self, forward_func) self._multiply_by_inputs = multiply_by_inputs @log_usage() def attribute( self, inputs: TensorOrTupleOfTensorsGeneric, neuron_selector: Union[int, Tuple[Union[int, slice], ...], Callable], - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - captum/attr/_core/neuron/neuron_integrated_gradients.py [14:76]: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - r""" Approximates the integral of gradients for a particular neuron along the path from a baseline input to the given input. If no baseline is provided, the default baseline is the zero tensor. More details regarding the integrated gradient method can be found in the original paper here: https://arxiv.org/abs/1703.01365 Note that this method is equivalent to applying integrated gradients where the output is the output of the identified neuron. """ def __init__( self, forward_func: Callable, layer: Module, device_ids: Union[None, List[int]] = None, multiply_by_inputs: bool = True, ) -> None: r""" Args: forward_func (callable): The forward function of the model or any modification of it layer (torch.nn.Module): Layer for which attributions are computed. Output size of attribute matches this layer's input or output dimensions, depending on whether we attribute to the inputs or outputs of the layer, corresponding to attribution of each neuron in the input or output of this layer. Currently, it is assumed that the inputs or the outputs of the layer, depending on which one is used for attribution, can only be a single tensor. device_ids (list(int)): Device ID list, necessary only if forward_func applies a DataParallel model. This allows reconstruction of intermediate outputs from batched results across devices. If forward_func is given as the DataParallel model itself, then it is not necessary to provide this argument. multiply_by_inputs (bool, optional): Indicates whether to factor model inputs' multiplier in the final attribution scores. In the literature this is also known as local vs global attribution. If inputs' multiplier isn't factored in then that type of attribution method is also called local attribution. If it is, then that type of attribution method is called global. More detailed can be found here: https://arxiv.org/abs/1711.06104 In case of Neuron Integrated Gradients, if `multiply_by_inputs` is set to True, final sensitivity scores are being multiplied by (inputs - baselines). """ NeuronAttribution.__init__(self, forward_func, layer, device_ids) GradientAttribution.__init__(self, forward_func) self._multiply_by_inputs = multiply_by_inputs @log_usage() def attribute( self, inputs: TensorOrTupleOfTensorsGeneric, neuron_selector: Union[int, Tuple[Union[int, slice], ...], Callable], - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -