Add multiple columns to PySpark dataframe
I solved the task with the help of rdd.mapPartitions. This seems to be the best solution.
For every dataframe row I need to make a REST call and use response in order to create multiple columns in the dataframe.
I see the following nasty solution:

add temporary column response which will contain the response
build needed columns from response
finally remove response

What is the proper way to solve my problem?