Why should we modify the voltage source value if the resistor is connected with it directly?
This problem is about determining the Thevenin equivalent of a voltage source with some external resistors. 

In general, the Thevenin voltage will be the open-circuit voltage of the network you're simplifying.

In the first case, the open-circuit voltage is exactly equal to the original source voltage, because the output terminals are directly across the source.

In the second case, the open-circuit voltage is less than the original source voltage because the output is taken by a voltage divider formed from R4 and R1.