Neural Network for bit operation AND
This is probably not the best place to ask this question but I am really struggling. 
I want to create a "trivial" neural network which has 2 inputs, 3 hidden neurons and 1 output. The idea is that we feed it two booleans and it outputs the AND result. 

I wanted to do it in a somewhat "conventional" way (I am still very new to pytorch). I'm using the Adam optimiser with a learning rate of 0.001 and the nll_loss function (although I need to change it as I need to have only 1 output and this requires the number of outputs to match the number of classes - in this case 1 and 0)

I know this should be a really trivial problem but I am really struggling and can't find anything useful on google.

Make a .csv file with your training data.

x1  x2  label
TRUE    FALSE   FALSE
TRUE    TRUE    TRUE
FALSE   FALSE   FALSE
FALSE   TRUE    FALSE


Create a dataset class.

class boolData(Dataset):
    def __init__(self, csv_path):
        self.label = pd.read_csv(csv_path)

    def __len__(self):
        return len(self.label)

    def __getitem__(self, idx):
        sample = torch.tensor(self.label.iloc[idx,0:2]).int()
        label = torch.tensor(self.label.iloc[idx,2]).int()
        return sample, label

tensor_dataset = boolData(csv_path='sample_bool_stack.csv')

boolDL = DataLoader(tensor_dataset, batch_size=4, shuffle=True)

batch, labels = next(iter(boolDL))

batch, labels


(tensor([[1, 0],
         [1, 1],
         [0, 1],
         [0, 0]], dtype=torch.int32), tensor([0, 1, 0, 0], dtype=torch.int32))


Initialize embeddings.

def _emb_init(x):
    x = x.weight.data
    sc = 2/(x.size(1)+1)
    x.uniform_(-sc,sc)


The model converts True, False to integers and takes the sum as the input.
EDIT: output with just single neuron.

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.emb = nn.Embedding(3, 2)
        _emb_init(self.emb)
        self.fc1 = nn.Linear(2, 10)
        self.fc2 = nn.Linear(10, 1)

    def forward(self, x):
        x = torch.sum(x, dim=1)
        x = self.emb(x)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.fc2(x)
        return x


Initialize model, optimizer, and loss function. EDIT: change loss to match single neuron output.

model = Net()
opt = optim.SGD(model.parameters(), lr=1e-3)
loss_func = nn.BCEWithLogitsLoss()
num_epochs = 10000

device = "cpu"


Train model.

for epoch in range(num_epochs):
    for i,(inputs, labels) in enumerate(boolDL):
        inputs = inputs.to(device).long()
        labels = labels.to(device).float()

        opt.zero_grad()

        output = model(inputs)
        loss = loss_func(output.view(-1), labels)
        loss.backward()
        opt.step()
        with torch.no_grad():
            if epoch % 2000 == 0: print(loss.item())


0.004416308831423521
0.002891995944082737
0.004371378570795059
0.0017852336168289185


Test model.

inputs = torch.tensor([[0,0],[0,1],[0,0]])

def check_model(inputs):
    out = model(inputs)
    preds = torch.sigmoid(out).round()
    return preds.detach().numpy()

check_model(inputs)


array([[0.],
       [0.],
       [0.]], dtype=float32)

