AttributeError: module 'torch.distributed' has no attribute 'is_initialized' in pytorch==1.11.X
whenever we are creating the object for TrainingArguments
from transformers import Trainer, TrainingArguments
batch_size = 64
logging_steps = len(emotions_encoded[&quot;train&quot;]) // batch_size
model_name = f&quot;{model_ckpt}-finetuned-emotion&quot;
training_args = TrainingArguments(output_dir=model_name,
                                      num_train_epochs=2,
                                      learning_rate=2e-5,
                                      per_device_train_batch_size=batch_size,
                                      per_device_eval_batch_size=batch_size,
                                      weight_decay=0.01,
                                      evaluation_strategy=&quot;epoch&quot;,
                                      disable_tqdm=False,
                                      logging_steps=logging_steps,
                                      push_to_hub=True,
                                      log_level=&quot;error&quot;)

I am getting
AttributeError: module 'torch.distributed' has no attribute 'is_initialized'

In order to solve this problem
Actually Window and Mac doesn't support distributed training facility
so this issue is occuring
To solve this problem
go to your transformers package where you install it
in my case it is
Desktop/rajesh/pytorch_env/env/lib/python3.8/site-packages/transformers/training_args.py

replace line-1024
if torch.distributed.is_initialized() and self.local_rank == -1:
with
if True and self.local_rank == -1:
Restart your kernal
