DataFrame view in PyCharm when using pyspark
Pycharm does not support spark dataframes, you should call the toPandas() method on the dataframe. As @abhiieor mentioned in a comment, be aware that you can potentially collect a lot of data, you should first limit() the number of rows returned.
I create a pyspark dataframe and i want to see it in the SciView tab in PyCharm when i debug my code (like I used to do when i have worked with pandas).
It says "Nothing to show" (the dataframe exists, I can see it when I use the show() command).
someone knows how to do it or maybe there is no integration between pycharm and pyspark dataframe in this case?