How does a language abstract away from underlying byte machines?
I'm assuming you are really asking 


  how is it possible that the same source code can be compiled to different architectures?  


In order to achieve this feat different systems use different solutions.

Interpreter
Some languages use an interpreter. In this case there is no underlying architecture. The source code gets interpreted as written.

Virtual machine using byte code
Languages like C# and java compile to intermediary code (called byte code) for a virtual machine.
A runtime then executes this code somewhat like an interpreter would.
Frequently a just-in-time compiler further compiles (critical parts of) the byte-code to actual machine code.
So Java and C# do not care about the actual machine.  

Virtual machine as intermediary target
Other compilers use the LLVM architecture. LLVM presents a virtual machine to the front end and a compiler using LLVM compiles for this virtual machine.
Specific LLVM backends tuned to a specific CPU architecture further compile this intermediary code to actual machine code.
LLVM differs from Java in that LLVM always compiles all code to machine code and it does not force any specific runtime upon the compiler.

Direct compilation
Traditional compilers like gcc (gnu C) compile the code to a specific architecture in a direct manner. Every architecture requires a different compiler which follows the language rules to create assembly code.
Maintaining separate compilers for different CPU architectures whilst ensuring all those abstractions work the same on all targets is accomplished through sheer hard work and coordination between compilers.

Hence the rise of intermediary virtual machines as compilation targets.   


  To what extend does a compiler abstract away the underlying CPU?  


This differs per language. Older languages like C tend to less abstraction.
The trend in newer languages is towards more abstraction.
Some languages offer both abstract types (always the same irrespective of architecture) as well as implementation dependent types (different for different architectures).
An example of this is a NativeInt type that is 32 bits on X86 and 64 bits on X64.    

CPU's do some of the abstraction
The other element in play is that CPU's are converging towards a single 'constellation'. More and more CPU architectures are using the X64 model of 32 bit integers and 64 bit memory.
Also many CPU's are backwards compatible with their predecessors.
As long as a compiler produces code that runs on a current or previous version of a CPU that CPU will run it.
So in that case it's the CPU that abstracts away (some of) the differences.    


  ... Byte machine  


A byte machine does not exist.
There is byte code, which is code for the virtual machine used by e.g. Java.
and there is machine code, which is what a CPU consumes.    

Memory management
You talk about memory management.
This is a subject completely different from abstraction.
Every CPU architecture handles memory in essentially the same way.
All but the most esoteric CPU's have a stack and use pointer logic to address the heap.
The fact that e.g. Java and C# use a garbage collector is not technically abstraction, but relates to resource management.
How resource management gets done depends upon the runtime library in use by that language.  

Dynamic types
Dynamic types are invented to solve the problem of buffer overflows (arguably the number one security issue facing computing today). 
If the runtime library handles string and array management then the burden on the programmer decreases.  


  How does a languages abstract away managed or dynamic types 


By using a runtime library.
All usage of dynamic and managed types gets translated into calls to the runtime library.
E.g. when using a dynamic string, the runtime library handles the reference counting, copy-on-write and growing and shrinking the storage as needed.
None of these actions are visible in the source code.