When editing participants, can I avoid having all questions from all current events displayed?
You should be able to limit the custom participant data to a specific event or event type and then it shouldn't show up in the back end for events you didn't mean it to.

This is done on the custom data itself (under "used for") rather than in the profile. Also to note, you usually only want to 'require' data inside the profile not inside the custom data set.