And of course if you have just one input to process you then need to pass the same string to both gpus
If you use only one GPU, then you will have only rank 0.
rank = torch.distributed.get_rank()
if rank == 0:
    text_in = "Is this review positive or negative?