Can we read the data in a pickle file with python generators
Nope. The pickle file format isn't like JSON or something else where you can just read part of it and decode it incrementally. A pickle file is a list of instructions for building a Python object, and just like following half the instructions to bake a cake won't bake half a cake, reading half a pickle won't give you half the pickled object.
I have a large pickle file and I want to load the data from pickle file to train a deep learning model. Is there any way if I can use a generator to load the data for each key? The data is in the form of a dictionary in the pickle file. I am using pickle.load(filename), but I am afraid that it will occupy too much RAM while running the model. I used pickle.HIGHEST_PROTOCOL to dump the data to the pickle file initially.