show() for large number of columns in pyspark
the display() function is what you want.
If your dataframe is named df, you'll call the function as such:
display(df) .
I am working with a dataframe that was 20 columns. When I try to display output using .show() after manipulations, due to the large number of columns its not very easy to read. What's the best way to get a compact view of all columns?
Thanks
SS