def end_idx_last_full_word()

in simuleval/data/tokenizer.py [0:0]

• 6 lines of code
• 4 McCabe index (conditional complexity)

    def end_idx_last_full_word(self, tokens):
        # Begin of word indices
        bow_indices = [i for i, t in enumerate(tokens) if t[0] == '\u2581']

        if len(bow_indices) < 2:
            return 0
        else:
            return bow_indices[-1]