Replaced v-brake pads too late - how to check if rim is damaged?
One way to check is placing a straight edge, for example a ruler, against the braking surface. If the surface is worn enough to be concave, there will be a visible gap in the middle. 

Small scratches are not a problem, as already mentioned in a comment, many rims have pre-machined grooves to serve as wear indicators and even these do not cause any structural problems.