How can you update a pyfile in the middle of a PySpark shell session?
I don't think it's feasible during an interactive session. You will have to restart your session to use the modified module.
Within an interactive pyspark session you can import python files via sc.addPyFile('file_location').  If you need to make changes to that file and save them, is there any way to "re-broadcast" the updated file without having to shut down your spark session and start a new one?
Simply adding the file again doesn't work.  I'm not sure if renaming the file works, but I don't want to do that anyways.
As far as I can tell from the spark documentation there is only a method to add a pyfile, not update one.  I'm hoping that I missed something!
Thanks