Effective time constant of an RC circuit
I have given a lot of thought on this but I am not able to figure out why that method works.


This is an application of the Thevenin theorem. 

Step 2 is very straightforward. You're treating everything except the capacitor as a one-port network, and the capacitor as the load. Then you're following the usual method of finding the Thevenin resistance of the source network.

Step 3 is a bit less obvious, but you're actually doing the same thing. You treat everything except the resistor as a one-port source, zero the load, and find the equivalent impedance of the source. 

If you're familiar with Laplace analysis, you know that capacitors can be treated as a kind of complex-valued resistor, aka an impedance or reactance (in this case, where the impedance will be purely imaginary). Understanding the Laplace analysis makes it much more clear why the two scenarios (several resistors and one capacitor or several capacitors and one resistor) can be treated in very similar ways.


  Can't this method be somehow extended to the general case containing any number of capacitors and resistors?


In this case, you can pick out one capacitor and find a Thevenin equivalent source for the network driving it.

But the behavior won't necessarily be equivalent to a single resistor in series with this capacitor, allowing you to characterize the circuit with a single time constant.