is endogeneity a problem in the first-stage regression in a two-stage least square regression?
Yes it is a problem. The first stage itself has to satisfy the same assumptions that standard OLS would and $cov(Z,\epsilon_1)\neq 0$ would violate them (see A Guide to Modern Econometrics by Verbeek).
Furthermore, actually the two conditions you mention are not enough. The instrument should also not be 'weak', that is the first stage should have $F$-statistics above $10$ (as a rule of thumb). Also the instrument should have an effect on $Y$ only through $D$ - the so called exclusion restriction (see Mostly Harmless Econometrics by Angrist and Pischke).