Blender Bevel can not bevel a simple all-quad geometry! Bevel modifier has no problem giving a beautiful result for a mesh with a bunch of N-gons (The outer mesh) but cannot properly bevel a simple all-quad mesh (The inner mesh)?!
I have found a solution. Turn out all I have to do is to flip normal of faces at the area with the problem.