Is it suitable to find inverse of last layer's activation function and apply it on the target output?
The main difference your change would have is to allow you to apply a loss function to a different part of the network. This may affect training.

If you keep the same loss function (e.g. MSE), but apply it to the pre-transformed values, then you will have changed the objective of the network, perhaps significantly. Whether or not this is a good thing depends on how much you needed the original loss function. However, the fact that it would result in a different training target is usually going to be a bad thing if your original training target was correct. This will also be true if you pick a new arbitrary loss function that seems to fit the pre-transform representation better.

If you engineer a "correct" loss function such that the objective of the network remains unchanged, then the behaviour of the network will not change much - probably not at all. However, in some cases this can lead to more stable and/or faster training - it is often used for classifiers to avoid need to use exponentiation, see tf.nn.softmax_cross_entropy_with_logits in TensorFlow, which does exactly this.