How to work with large amount of data overcoming RAM issues in python I use numpy arrays to work with deep learning images. But as the data gets bigger, I'm facing issue with RAM even before training the model when using techniques like data augmentation.

Can someone suggest me how to work with large data for eg. 30GB of data in my system which has 16gb ram.

P.S. I'm worried about RAM during preprocessing and training, while i do batch processing with my GPU
Dask is designed to manage these types of workloads. It provides an interface like NumPy, Pandas, or Python iterators for larger-than-memory operations. An example of using Dask with TensorFlow can be found here.