Will changing a /system partition file system delete all data? I am trying to remove forgotten PIN number on my Samsung Galaxy S3 without removing any data. 

I already installed custom recovery TWRP, but cannot mount /system partition -> Invalid argument error. I found some tutorials that suggest changing /system file system to FAT and then back to original file system, to be able to mount.

I am affraid this will remove all data and I want be able to use my phone.

Is this correct?

The phone is not rooted, but error is not about permissions, but about some argument.
It highly depends on how you change the partition FS and what software you use to do it.

If I guess it right, the FS on your /system partition is Linux ext4, which does not have a lossless way to be converted to FAT that I am aware of, so I guess it's mostly formatting it twice: to FAT, then back.

It's obvious that formatting will make all data on that partition lost. However, if you find, or if the tutorial instructs, another way to losslessly convert ext4 to FAT, it's possible that you can change the FS without losing any data. But be aware that changing ext4 to FAT will mess up all the permissions on files, which may potentially lead to extra work before it turns into a usable state.

Proceed with caution.