Similar objects same labels
If you only need to predict whether an object is a pepper or not why bother predicting whether it is a red pepper or a green pepper, you are making your problem more complicated than it needs to be. In my opinion you should try to make the problem you want to solve as simple as possible so I would advise using the shortest label possible as long as it solves your problem.