Is NN with no hidden layer is behave like a regression?
You would then call it perception model. As in 1945 or so some researcher did this with no hidden layer. They came up with AND,OR,XOR problem where you can solve AND,OR with a single perception model but that model was unable to solve the XOR problem.

There are two kinds of regression: Linear and non linear. The activation functions in neural net makes them non linear regressor.

A Neural Net would be a linear regressor if you don't put any activation function.

Then it would be like : $Wx+b$ (which is $y=mx+b$ equivalent).
But when you add activation function let it be sigmoid then it achieves a non linear space.Like $$\frac{1}{1+e^{-(wx+b)}}$$

"If we have for instance 20 input and 4 output and I have no true label, is it similar to regression?" --- In this you are just deleting back propagation thing. what as usual regression algorithms like linear,polynomial does is they just predict the function but never learns something like neural net learns proper function via optimizing weights. If you don't have true level that means you are not calculating the loss value between [true-predicted] so it would be a simple regression.