def allreduce()

in utils.py [0:0]

• 4 lines of code
• 3 McCabe index (conditional complexity)

def allreduce(x, average):
    if mpi_size() > 1:
        dist.all_reduce(x, dist.ReduceOp.SUM)
    return x / mpi_size() if average else x