Usually I achieve low coupling by creating classes that exchange lists, sets, and maps between them. Now I am developing a  Java batch application and I can't put all the data inside a data structure because there isn't enough memory. I have to read and process one chunk of data and then going to the next one. So having low coupling is much more difficult because I have to check somewhere if there is still data to read, etc.What I am using now is:Source -> Process -> PersistThe classes that process have to ask to the Source classes if there are more rows to read.What are the best practices and or useful patterns in such situations?I hope I am explaining myself, if not tell me.
Low coupling processing big quantities of data
java;design patterns;programming practices;patterns and practices;coupling
From the comments I see you're using Java. Have a look at various Queue implementations. Particularly, BlockingQueue is useful for producer-consumer scenarios. You could have two queues: one between Source (producer of data) and Process (consumer of data), and another between Process (producer of results) and Persist (consumer of results).With limited-capacity blocking queues it's fairly easy to implement efficient (the bottleneck part, whatever it is, is kept fed with data 100% of the time) systems, still using only a bound amount of memory no matter how much data there is.