Problem with getting handrail flange bracket tight to post
I am inclined to agree with Jack in the initial comments here that the bracket shown is what not going to ever be able to constrain the post to no movement. At best the depth of engagement of the post into the bracket is about two inches and the total length of the post is probably about 36 to 40 inches. Think of the size of lever arm at play here. It is very easy to see how even moderate pressure on the post could deform the bottom portion. This would be even more true if the post itself is relatively thin material.

Even if the post were literally welded to the bracket any racking of the post would likely experience problems with the relatively small studs that you have embedded into the concrete.

You will have to find a way to secure the upper end of the post in such manner as to constrain it from movement. This will undoubtedly be taken care of in a direction that is inline with railing assembly once things are fully assembled. Racking of the post in a direction that is at right angles to the railing will require additional measures. Some possible options:


If the upper portion of the railing is adjacent to the building structure use another type of bracket to bolt the upper portion of the post to the building.
If there is additional span of the base concrete about 12 to 18 inches from the post on the side away from where you walk you could attach another bracket to the concrete and secure a bar or rail from there at an angle up about half  the way up the post. This will create a strong triangular structure to prevent the racking. It even looks as if Lowes sells a  bracket made for just this purpose: