Do rivers/creeks flow underground when they dry up at the surface?
Water in liquid state always takes the fastest route to lower altitude. All rainwater would flow vertically down to the groundwater surface if not the permeability was limited by the material properties. 

The permeability controls how fast water infiltrate and penetrate the material. For example, can surface runoff water flow on top of a hard road coating, while water from a watering-can goes into the soft soil in the flower-bed. Gravel, sand and fractured rocks have high permeability whiles shale and non-fractured igneous rock have low permeability. So low, that in many cases it appears to be totally impenetrable on a human time scale. 

The same goes for a river. If the river flows on permeable rocks, typical sediments deposited by the river itself (see picture), ground water will also flow in the sediments under the river bed, permitting that water is still entering upstream. 

A better way to understand a river or lake in this context, is that it is the surface manifestation of the groundwater table. If you dig a well next to a body of water you’ll reach ground water at the same altitude as the river water surface. The gradient of the groundwater surface and the permeability of the material controls the direction and velocity of the flow. 

However, if more water enters a system than the surface can infiltrate, the surplus will flow as surface runoff, typical for wadis (See picture) in desert areas. 



So, to answer your question. It depends on the local geology, but ground water will always flow from the precipitation area to the oceans (or a lower point where it can evaporate or being pumped up). Ground water exists everywhere and the gradient of the hydrostatic pressure controls the direction of the flow. As viable rivers shows the ground water table, it is usually rather safe to assume that the ground water under the dry river flows more or less the same direction. 

However, ground water flow can be changed by e.g. pumping from irrigation wells. Imagine a flat lying river bed with a well upstream. When enough water is removed from the well and eventually evaporated, ground water will flow towards the well. The water don't 'remember', but follows the law of gravity.