even when you cure the memory issuesyou may need to deal with another limited resourcetime although computer may think you live for millions of yearsin reality you won' (unless you go into cryostasis until your pc is donecertain algorithms don' take time into accountthey'll keep running forever other algorithms can' end in reasonable amount of time when they need to process only few megabytes of data third thing you'll observe when dealing with large data sets is that components of your computer can start to form bottleneck while leaving other systems idle although this isn' as severe as never-ending algorithm or out-of-memory errorsit still incurs serious cost think of the cost savings in terms of person days and computing infrastructure for cpu starvation certain programs don' feed data fast enough to the processor because they have to read data from the hard drivewhich is one of the slowest components on computer this has been addressed with the introduction of solid state drives (ssd)but ssds are still much more expensive than the slower and more widespread hard disk drive (hddtechnology general techniques for handling large volumes of data never-ending algorithmsout-of-memory errorsand speed issues are the most common challenges you face when working with large data in this sectionwe'll investigate solutions to overcome or alleviate these problems the solutions can be divided into three categoriesusing the correct algorithmschoosing the right data structureand using the right tools (figure problems choose the right algorithms solutions choose the right data structures choose the right tools handling large data general tips figure overview of solutions for handling large data sets no clear one-to-one mapping exists between the problems and solutions because many solutions address both lack of memory and computational performance for instancedata set compression will help you solve memory issues because the data set becomes smaller but this also affects computation speed with shift from the slow hard disk to the fast cpu contrary to ram (random access memory)the hard disc will store everything even after the power goes downbut writing to disc costs more time than changing information in the fleeting ram when constantly changing the informationram is thus preferable over the (more durablehard disc with an