steel poles with pine shelving - how to fix?
Your problem is there's nothing holding the 4 posts in place.  The fix is to attach the shelves to the posts as tightly as possible - they can't just set on the posts.  What you've basically done is tried to create a large version of these common steel shelves:



If you've ever put one together, the strength and stability comes from the wedge anchors on each corner holding the shelves tightly in place.

Your unit's stability is proportional to how tight you can attach each shelf to each corner.  Any looseness will be notices in wobble.

I'm not familiar with the brackets you say that you are using, but you could possibly sandwich the shelves between two brackets (one on top, one on bottom, but they'd have to make a very tight sandwich).  Or if it's like a pipe flange maybe the shelf can be securely screwed to the brackets.  The point is, you have to have a tight, secure connection on each corner of each shelf.