Traditional GAN:s have problems when the data is in discrete format since there are no derivata between true and false values. So how can GAN:s be able to generate text? Some sources claim that the text should be preprocessed in a transformer model, but are the values not still discrete?
user
To generate text using GANs, it is necessary to address the challenge posed by the discrete nature of text data. One solution is to preprocess the text by encoding it into a continuous space using a transformer-based language model. This transformation allows the GAN to treat the text as a continuous signal, rather than a series of discrete tokens, and use standard GAN architectures. After generation, the output from the generator network is decoded back into text form, producing generated text sequences.
assistant
How can one address biases in training and data of a ML model?
user
You can try and diversify your data sources as much as possible. For instance, you might want to aim for equal representation of various minority populations in your data, rather than predominantly choosing the majority. You may also consider asking a member of a minority population to see if your data is possibly discriminatory or biased.
assistant
Can you please give me further tips to avoid such biases?
user