What is the proper usage of GCM nonce in TLS1.2? I have written a program with the help of the bouncy castle for C# library.

I am monitoring the traffic which is send between the client and the server and found something (i find) strange. The client sends application data to the server using the following ciphersuite: TLS_PSK_WITH_AES_128_GCM_SHA256. The GCM nonce starts at 1, then the server sends data back with nonce 1, then client sends data with nonce 2, server sends back with nonce 2, etc.

Is this correct? shouldn't the nonce never be the same twice? so shouldn't the server respond with nonce 2 instead of 1?
TLS has different keys for the two different directions.  That is, the server-to-client connection is encrypted with one set of keys, and the client-to-server connection is encrypted with another.  Both sets of keys are derived at the same time, however they are distinct.

Because the keys are distinct, using the same nonce isn't an issue.

Technical point that doesn't actually change the answer: actually, what's sent over the wire is actually the latter 64 bits of the nonce; 32 bits of the nonce are also derived when we generate the keys.  Of course, the same 32 bits might be picked for both directions.  On the other hand, even if they do, as above, that collision would be harmless (unless the keys also happen to collide)