def get_size_per_example()

in utils/hf_dataset_subsampling.py [0:0]

• 4 lines of code
• 2 McCabe index (conditional complexity)

def get_size_per_example(texts: List[str]) -> Dict:
    size_values = [len(text.encode()) for text in texts]
    examples = {"bytes_len": size_values}
    return examples