Should I keep specific distance from Class B?
You should remain at least 500 feet vertical separation from the Class B airspace, considering your direction of flight in accordance with FAR 91.159 (proper VFR cruising altitude) if you are more than 3000 feet AGL.  Or 2-3 miles lateral separation from Class B airspace (no regulation, just common sense).

If the Class B floor base altitude of 5000 feet msl is at or less than 3000 feet AGL, you could legally fly up to just below the base of the Class B and therefore come dangerously close to other aircraft operating at 5000 feet msl within the Class B.  This is too close for safety (considering allowable altimeter error, you could actually be at the same true altitude as an aircraft flying in the Class B).

Bottom line, in my opinion, you should always be at least 500 feet vertically or 2-3 miles laterally from Class B airspace.

This is what the Aeronautical Information Manual states regarding Class B:

(see the AIM):AIM 3-2-2