How do I convert 'the array saved as string to csv' back to float array?
I had to merge a lot of files (containing word embeddings and othe real valued vectors) based on some common attributes so I used Pandas DataFrame and saved the intermediate files as csv.

Currently I have a dataframe whose columns look something like this: 



I want to merge all last 4 columns (t-1embedding1a,t-1embedding7b,t-2embedding1a,t-2embedding7b) into a single vector to pass to neural network. 

I planned to iterate over the current dataframe and take 4 temporary tensors with value of each column and concatenate and write to new dataframe.
However torch.tensor doesn't work as it says:

torch_tensor = torch.tensor(final['t-1embedding1a'].astype(float).values)

could not convert string to float: '[-6.12873614e-01 -5.58319509e-01 -9.73452032e-01  3.66993636e-01\n  



I also tried np.fromstring() but the original values are lost in this case.

Sorry, if the question is unnecessarily complicated, I am a newbie to pytorch. Any help is appreciated!

First of all, the data type for columns with "t-lembeddingXX" is string that look like "[-6.12873614e-01 -5.58319509e-01 -9.73452032e-01 3.66993636e-01]". You have to convert them to a list of float.

final["t-lembeddingXX"] = final["t-lembeddingXX"].apply(lambda x : [float(x) for x in x.replace("[", "").replace ("]", "").split()])


Then, you have to check that each list of final.loc[i,"t-lembeddingXX"] has the same lengths.

If I haven't mistaken, you want to merge the 4 columns into one verctor.

all_values = list(df["t-lembeddingX1"]) + list(df["t-lembeddingX2"]) + list(df["t-lembeddingX3"]) + list(df["t-lembeddingX4"])
# there is sureliy a better way


Then pass to tensor:

torch_tensor = torch.tensor(all_values)


------------

Finally, I advise you to take a look at the function of torch.cat. You can convert each column to a vector and then use this function to concatenate them together.
