Wide to long format pandas
Have a look at pandas.melt function. It might do the trick for you
Desired Output -




id
gender
random_genre




1
F
fiction


1
F
history


2
M
mystery


3
F
fiction


4
M
history




I have a dataframe where a user can have multiple preferences and I have to join these with some other kind of preferences which makes the table exponentially large. Like x, y etc. preferences are up to 80+ of one kind and almost 40 are of another kind.
Until now I was pivoting the table (pivot_table) and performing merge but I want to use the output for charts (count of a preference etc.).
Current output after pivot_table -




id
gender
fiction
history
mystery




1
F
1
NaN
NaN


1
F
NaN
2
NaN


2
M
NaN
NaN
3


3
F
1
NaN
NaN


4
M
NaN
2
NaN




Having almost 80+ preferences of just one type and more after joining.
How can I convert all these back after joins to the first table, where I just have a single column for all the preferences and if an id has multiple preferences a new row is created for the same?