Why is the Z-transform so important in digital filters analysis and design?
The Z Transform is to discrete-time (digital) signals precisely the same role that the Laplace Transform is to continuous-time (analog) signals.

Linear Time-Invariant (LTI) Systems (a.k.a. "filters"), are made up of signal-processing elements that fall into 3 fundamental classes:


adders (devices that add two signals).
scalers (devices that scale a signal by a constant).
"reactive" elements (devices that are able to discriminate w.r.t. frequency).


Element classes 1. and 2. are essentially the same for analog or digital filters.  The are sometimes called "memoryless" devices or elements.

For an analog filter (or "analog LTI system"), those reactive elements would be capacitors or inductors.  They integrate ($s^{-1}$) one signal to become another.  That turns a sine signal into a cosine signal or shifts the phase by $\pm$ 90°.

$$\cos(\Omega t) = \sin(\Omega t + \tfrac{\pi}{2})$$

For digital filters, the reactive elements are delay elements.  A unit delay (a delay of exactly one sample period $T$) will delay any signal, including a sinusoid, by 1 sample or $T$ units of time ($z^{-1}$).  That shifts the phase by an amount that is dependent on frequency

$$\sin(\Omega (t-T) ) = \sin(\Omega t - \Omega T)$$

or

$$\sin(\omega (n-1) ) = \sin(\omega n - \omega )$$

Any LTI system that acts as a "filter", a device to filter out some frequency components and leave others, must have reactive elements (or "non-memoryless" elements or components having memory) in order to discriminate one frequency from another.  And such a filter will shift phase which will normally be different for different frequencies.  But a memoryless LTI system (which is just a scaler) will not discriminate between frequencies nor will shift phase, except for possibly by 180°, which is just a polarity reversal or scaling by a negative constant.