Autoencoder to encode features/categories of data
My question is regarding the use of autoencoders (in PyTorch). I have a tabular dataset with a categorical feature that has 10 different categories. Names of these categories are quite different - some names consist of one word, some of two or three words. But all in all I have 10 unique category names. What I'm trying to do is to create an autoencoder which will encode names of these categories - for example, if I have a category named 'Medium size class', I want to see if it is possible to train autoencoder to encode this name as something like 'mdmsc' or something like that. The use of it would be to found out which data points are hard to encode or not typical or something like that. I tried to adapt autoencoder architectures from various tutorials online however nothing seems to work for me or I simply do not know how to use them as they are all about images. Maybe someone has any idea how this type of autoencoder might be accomplished if it is at all possible?

Edit: here's the model I have so far (I just tried to adapt some architectures I found online):

class Autoencoder(nn.Module):

def __init__(self, input_shape, encoding_dim):
    super(Autoencoder, self).__init__()

    self.encode = nn.Sequential(
        nn.Linear(input_shape, 128),
        nn.ReLU(True),
        nn.Linear(128, 64),
        nn.ReLU(True),
        nn.Linear(64, encoding_dim),
    )

    self.decode = nn.Sequential(
        nn.Linear(encoding_dim, 64),
        nn.ReLU(True),
        nn.Linear(64, 128),
        nn.ReLU(True),
        nn.Linear(128, input_shape)
    )

def forward(self, x):
    x = self.encode(x)
    x = self.decode(x)
    return x

model = Autoencoder(input_shape=10, encoding_dim=5)


And also I use LabelEncoder() and then OneHotEncoder()to give these features/categories I mentioned numerical form. However, after training, output is the same as was input (no changes on the category name) but when I try to use only encoder part I'm unable to apply LabelEncoder() and then OneHotEncoder() because of dimension issues. I feel like maybe I can do something differently at the beginning, then I try to give those features numerical form, however I'm not sure what should I do.

First you will need to set up a train_loader depending on your data that will iterate over your data points.

Then you need to figure out what kind of loss you are going to use and optimizer:

# mean-squared error loss
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=.001)  #learning rate depend on your task


Once you have that ready, you can train your autoencoder with basic steps:

for epoch in range(epochs):
        for features in train_loader:

            optimizer.zero_grad()
            outputs = model(batch_features)
            train_loss = criterion(outputs, features)
            train_loss.backward()
            optimizer.step()


Once model is done training you can examine embeddings using:

embedding = model.encode(your_input)

