What are some possible "functional" memory structures? With my thin knowledge on embedded systems, compilers, and computer architectures, I know that the basics of computer memory(physical) are sort of like an array, with addressing which work like pointers. I see that imperative programming(which eventually unwraps to assembly, or the Turing Machine) fits this architecture very well.

I am a bit interested to see if there is a counterpart of physical memory that might fit the functional programming style. By "fitting", I am making the assumption that all pure functional languages can unwrap to be immutable lambda expressions(via Lambda Calculus). Now, with the array-like memory structure, we have to allocate and gc objects under the scene in order to get lambdas to work(because lambda expressions do not address operations on memory directly). Just as a thought experiment, is it possible to have a device that fits the need of memory in lambda expressions, while not containing a garbage collector?

Yes, I know that some imperative programming languages do use garbage collectors. However, when boiled down to assembly, it doesn't. So I am more curious about the evaluation of "pure" lambda expressions.

Just as a note, the lambda expressions I talk about include the fixed point operator(i.e. recursion).
The topic is fairly broad. 

Functional hardware has been explored in e.g. Lisp machines.

More on the theoretical side, you may want to read about the various abstract machines which have been proposed for running functional languages.
Some of them are the SECD machine, the CAM (categorical), the Krivine machine, the STG machine.

Perhaps a more radical approach can be learnt by looking at the various graph-reduction machines. A pioneering work by Lamping describes how lambda can be run faster by rewriting graphs (!), instead of expressions.
More recently, linear logic simplified the several rewriting rules.

Some of these approaches can give you some hint at how a custom functional hardware should be made to speed up execution.

For conventional hardware, the STG machine seems one of the most promising approaches.