captum/influence/_core/tracincp.py [379:472]: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ) -> None: r""" Args: model (torch.nn.Module): An instance of pytorch model. This model should define all of its layers as attributes of the model. influence_src_dataset (torch.utils.data.Dataset or torch.utils.DataLoader): In the `influence` method, we either compute the influence score of training examples on examples in a test batch, or self influence scores for those training examples, depending on which mode is used. This argument represents the training dataset containing those training examples. In order to compute those influence scores, we will create a Pytorch DataLoader yielding batches of training examples that is then used for processing. If this argument is already a Pytorch Dataloader, that DataLoader can be directly used for processing. If it is instead a Pytorch Dataset, we will create a DataLoader using it, with batch size specified by `batch_size`. For efficiency purposes, the batch size of the DataLoader used for processing should be as large as possible, but not too large, so that certain intermediate quantities created from a batch still fit in memory. Therefore, if `influence_src_dataset` is a Dataset, `batch_size` should be large. If `influence_src_dataset` was already a DataLoader to begin with, it should have been constructed to have a large batch size. checkpoints (str or List of str or Iterator): Either the directory of the path to store and retrieve model checkpoints, a list of filepaths with checkpoints from which to load, or an iterator which returns objects from which to load checkpoints. checkpoints_load_func (Callable, optional): The function to load a saved checkpoint into a model to update its parameters, and get the learning rate if it is saved. By default uses a utility to load a model saved as a state dict. Default: _load_flexible_state_dict layers (List of str or None, optional): A list of layer names for which gradients should be computed. If `layers` is None, gradients will be computed for all layers. Otherwise, they will only be computed for the layers specified in `layers`. Default: None loss_fn (Callable, optional): The loss function applied to model. There are two options for the return type of `loss_fn`. First, `loss_fn` can be a "per-example" loss function - returns a 1D Tensor of losses for each example in a batch. `nn.BCELoss(reduction="none")` would be an "per-example" loss function. Second, `loss_fn` can be a "reduction" loss function that reduces the per-example losses, in a batch, and returns a single scalar Tensor. For this option, the reduction must be the *sum* or the *mean* of the per-example losses. For instance, `nn.BCELoss(reduction="sum")` is acceptable. Note for the first option, the `sample_wise_grads_per_batch` argument must be False, and for the second option, `sample_wise_grads_per_batch` must be True. Also note that for the second option, if `loss_fn` has no "reduction" attribute, the implementation assumes that the reduction is the *sum* of the per-example losses. If this is not the case, i.e. the reduction is the *mean*, please set the "reduction" attribute of `loss_fn` to "mean", i.e. `loss_fn.reduction = "mean"`. Default: None batch_size (int or None, optional): Batch size of the DataLoader created to iterate through `influence_src_dataset`, if it is a Dataset. `batch_size` should be chosen as large as possible so that certain intermediate quantities created from a batch still fit in memory. Specific implementations of `TracInCPBase` will detail the size of the intermediate quantities. `batch_size` must be an int if `influence_src_dataset` is a Dataset. If `influence_src_dataset` is a DataLoader, then `batch_size` is ignored as an argument. Default: 1 sample_wise_grads_per_batch (bool, optional): PyTorch's native gradient computations w.r.t. model parameters aggregates the results for a batch and does not allow to access sample-wise gradients w.r.t. model parameters. This forces us to iterate over each sample in the batch if we want sample-wise gradients which is computationally inefficient. We offer an implementation of batch-wise gradient computations w.r.t. to model parameters which is computationally more efficient. This implementation can be enabled by setting the `sample_wise_grad_per_batch` argument to `True`, and should be enabled if and only if the `loss_fn` argument is a "reduction" loss function. For example, `nn.BCELoss(reduction="sum")` would be a valid `loss_fn` if this implementation is enabled (see documentation for `loss_fn` for more details). Note that our current implementation enables batch-wise gradient computations only for a limited number of PyTorch nn.Modules: Conv2D and Linear. This list will be expanded in the near future. Therefore, please do not enable this implementation if gradients will be computed for other kinds of layers. Default: False """ TracInCPBase.__init__( self, model, influence_src_dataset, checkpoints, checkpoints_load_func, loss_fn, batch_size, ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - captum/influence/_core/tracincp_fast_rand_proj.py [85:154]: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ) -> None: r""" Args: model (torch.nn.Module): An instance of pytorch model. This model should define all of its layers as attributes of the model. final_fc_layer (torch.nn.Module or str): The last fully connected layer in the network for which gradients will be approximated via fast random projection method. Can be either the layer module itself, or the fully qualified name of the layer if it is a defined attribute of the passed `model`. influence_src_dataset (torch.utils.data.Dataset or torch.utils.DataLoader): In the `influence` method, we either compute the influence score of training examples on examples in a test batch, or self influence scores for those training examples, depending on which mode is used. This argument represents the training dataset containing those training examples. In order to compute those influence scores, we will create a Pytorch DataLoader yielding batches of training examples that is then used for processing. If this argument is already a Pytorch Dataloader, that DataLoader can be directly used for processing. If it is instead a Pytorch Dataset, we will create a DataLoader using it, with batch size specified by `batch_size`. For efficiency purposes, the batch size of the DataLoader used for processing should be as large as possible, but not too large, so that certain intermediate quantities created from a batch still fit in memory. Therefore, if `influence_src_dataset` is a Dataset, `batch_size` should be large. If `influence_src_dataset` was already a DataLoader to begin with, it should have been constructed to have a large batch size. checkpoints (str or List of str or Iterator): Either the directory of the path to store and retrieve model checkpoints, a list of filepaths with checkpoints from which to load, or an iterator which returns objects from which to load checkpoints. checkpoints_load_func (Callable, optional): The function to load a saved checkpoint into a model to update its parameters, and get the learning rate if it is saved. By default uses a utility to load a model saved as a state dict. Default: _load_flexible_state_dict loss_fn (Callable, optional): The loss function applied to model. `loss_fn` must be a "reduction" loss function that reduces the per-example losses in a batch, and returns a single scalar Tensor. Furthermore, the reduction must be the *sum* or the *mean* of the per-example losses. For instance, `nn.BCELoss(reduction="sum")` is acceptable. Also note that if `loss_fn` has no "reduction" attribute, the implementation assumes that the reduction is the *sum* of the per-example losses. If this is not the case, i.e. the reduction is the *mean*, please set the "reduction" attribute of `loss_fn` to "mean", i.e. `loss_fn.reduction = "mean"`. Default: None batch_size (int or None, optional): Batch size of the DataLoader created to iterate through `influence_src_dataset`, if it is a Dataset. `batch_size` should be chosen as large as possible so that certain intermediate quantities created from a batch still fit in memory. Specific implementations of `TracInCPBase` will detail the size of the intermediate quantities. `batch_size` must be an int if `influence_src_dataset` is a Dataset. If `influence_src_dataset` is a DataLoader, then `batch_size` is ignored as an argument. Default: 1 vectorize (bool, optional): Flag to use experimental vectorize functionality for `torch.autograd.functional.jacobian`. Default: False """ TracInCPBase.__init__( self, model, influence_src_dataset, checkpoints, checkpoints_load_func, loss_fn, batch_size, ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -