How can I solve this pytorch two devices error
I ran into a problem with PyTorch:
Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking arugment for argument mat1 in method wrapper_addmm)
model = nn.Sequential(
        nn.Linear(622, 512),
        nn.ReLU(),
        nn.Linear(512, 256),
        nn.ReLU(),
        nn.Linear(256, 5),
    ).to(device)

    loss_fn = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)

    train_loader = Data.DataLoader(
        dataset=train_dataset,
        batch_size=32,
        shuffle=True,
        num_workers=0,
    )

    test_loader = Data.DataLoader(
        dataset=test_dataset,
        batch_size=100,
        shuffle=True,
        num_workers=0,
    )

    best_acc = 0
    best_model = model.cpu().state_dict().copy()
    # train_acc = 0
    # test_acc = 0
    for epoch in range(20):
        for step, (batch_x, batch_y) in enumerate(train_loader):
            batch_x = batch_x.to(device)
            batch_y = batch_y.to(device)
            print(batch_x)
            print(batch_x.device, 0)
            out = model(batch_x.to(device)).cuda()
            print(out.device, 1)
            loss = loss_fn(out, batch_y.long())
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            train_acc = np.mean((torch.argmax(out, 1) == batch_y).cpu().numpy())

            with torch.no_grad():
                for batch_x, batch_y in test_loader:
                    batch_x = batch_x.to(device)
                    batch_y = batch_y.to(device)
                    print(batch_x.device, 2)
                    out = model(batch_x)
                    print(batch_x.device, 3)
                    test_acc = np.mean((torch.argmax(out, 1) == batch_y).cpu().numpy())
            if test_acc &gt; best_acc:
                best_acc = test_acc
                best_model = model.cpu().state_dict().copy()

Can someone help explain that， I've been working on this all day....

Note that .to() has different behavior when applied to nn.Modules and to torch.tensors: while for torch.tensor .to(device) creates a copy of the tensor on the device, with nn.Module .to(device) operates in place.
In your code, you move your model to CPU:
best_model = model.cpu().state_dict().copy()

make sure you move the model back to device after moving it to cpu.
