Making an ANN for Polynomial Regression
If anyone stumbles upon this with any of the same queries, here are the answers:
1 - Sigmoid should work but it restricts the output between 0 and 1 and since this was a regression problem, you could design your problem to get a normalized output and scale it to the right number. Or just use ReLU.
2 - Yes. The type of activation on the output node changes the type of output you get.
3 - Of course it will. The derivative of the activation gradient is multiplied with the backpropagating global gradient to give the local gradient. Changing the activation function will change the derivative too.
4 - Maybe try to understand the Math behind backpropagation. It's basic chain rule and once you get it, you won't get confused like this.
I have an assignment to build an ANN for regression problems using Python from scratch without using any ML library. You might have guessed I am a true beginner at this and the process is a little confusing so I would really appreciate some help in answering the few questions that I have.
This is the basic training algorithm that I understand for training an ANN:

Forward prop for prediction
Backward prop to calculate errors
Calculate deltas for each weight using the errors
Accumulate deltas over a dataset iteration and calculate the partial gradient for each weight
Optimize weights using gradient descent

I hope the steps make sense and are okay. Here are a few questions that I have:

What activation function should I use? Sigmoid is probably not the answer.
Should an activation function be used on the single output node?
The formula to calculate errors for hidden layers in back-prop is δ(l) = Transpose[ϴ(l)] x δ(l+1) .* g`[z(l)] where l is the layer number and g`[z(l)] I believe is the derivative of the activation function usually taken as a(l) .* (1 - a(l)). Will this change as we use an activation function other than sigmoid?
Any errors I made or any important tip?

Apologies if the questions are very basic. I am a raw beginner at this.