How to implement Z-buffering? I understand the theory behind z buffering - that I have to check for each pixel if its depth value is less than the one on z buffer or not, if yes then draw else not - but I don't understand how would I actually get the z value of each pixel ?

At first I thought I will use the equation of the plane of the polygon I am drawing, but then I have to call the inverse of viewport transform for every pixel. Isn't this every inefficient ?
The values for each pixel in the Z-buffer are interpolated from the values at the corners of the triangle during rasterization. To make this work, the projection matrix produces depth values that are a function of the reciprocal of the original depth value in camera-space.

That is, the $Z'$ value in each transformed vertex is of the form $aZ + b$, where $a$ and  $b$ together determine the near and far planes, and the transformed $W'$ value is just the original $Z$. After doing the homogeneous divide by $W'$, the transformed $Z'$ becomes $a + b/Z$. By using the reciprocal of $Z$ in the depth value like this, you can then just interpolate linearly in screen space from the three corners to each pixel to get correct results. Since the last step produces floating point results, and most Z-buffers use unsigned integer values, you'll need to bias, scale and quantize before writing to the  actual buffer. 

I seriously recommend Eric Lengyel's book Mathematics for 3D Game Programming for the nitty-gritty details of all this kind of stuff, he also explains perspective-correct interpolation of vertex attributes and all the other math you'll need to know to write a rasterizer.