What if all passengers move from one side to the other?
You can get a significant rolling tendency if you shifted cargo or pax all to one side, but not enough to affect controllability; it would just be annoying.  This would apply to most airplanes.

Transport airplanes will have a lateral Center of Gravity range that has to be respected, along with a maximum fuel imbalance limit, to ensure controllability in some critical case.   In the CRJs, which are quite narrow, if everybody piled into the seats in one side, you (or the autopilot) would definitely notice it and have to correct with aileron, although it would be no problem trimming it out.  A really large fuel imbalance would be a much bigger problem however and might use up quite a bit of the available roll authority, so there are fuel imbalance limits in addition to lateral CG limits.

FAR 25 requires that the roll trim system provide the ability to trim out the rolling moment from max lateral CG offset in the most critical case - on takeoff, one engine inoperative, climbing at V2 (engine failure safety speed), with maximum allowable fuel imbalance. On the CRJs this uses up only about a third of the airplane's total roll authority (the RJ's roll trim just offsets the neutral point of the control circuit).