Do I have to bootstrap the dummy variables used for event dates in an event study?
One way is to rely on a simple block bootstrap procedure. That is, you can draw $n$ contiguous blocks of length $t^{bs}$ (both LHS and RHS variables) which result in an overall bootstrapped sample of length $T$, which should be identical to the length of your original sample.
Then repeat your estimation for each bootstrap sample and save your results, e.g. 10000 times. Thus, inference is based on the distribution of your coefficients.
From my personal experience, dummy variables often result in singular (or nearly singular) matrices within your estimation procedure. However, you can either manually add these observations to your bootstrap sample or alternatively discard all samples resulting in numerical errors.