What is the exact structure within the nodes of a hidden layer?
Let's go part-by-part.

For a NODE within a layer, if we have p inputs, we would multiply a 1xp coefficient matrix with a px1 input vector, correct?


In practice, for that LAYER, we would multiply a mxp coefficient matrix by a px1 input vector, where m is the number of nodes within that layer?

These two are correct. No issues here. The only differences may be in implementation, the logic here is sound.

If we have a bias term, then it would be a mx(p+1) coefficient matrix and a (p+1)x1 input vector?

This is correct. Intuitively, it is analogous to &quot;offest&quot; term in the equation of a line, y = mx + c. c here plays the same role as the bias.

In terms of customization, for most out of the box NNs, do we choose to have a bias term or not? Or for the most part, it automatically includes one.

Most out-of-the box NN include a bias term by default, since they do provide flexibility to the network, no matter how small. Practically, their utility varies with architecture and application. For example, Feed Forward networks have a good boost in performance with biases when used for regression. CNNs used for classification usually aren't affected much if biases are removed.

In terms of customization, this combination of inputs is always linear? There are no other ways to combine the inputs (such as including polynomials, interaction terms, etc...)

Non-linear combinations are definately possible. But using such combinations basically kills the purpose of ANNs - to have simple stacked operations approximate a complex function. Non-linear combinations may help in rare cases, but more often than not, will probably lead to underfit or overfit models. By convention, any non-linearity is introduced into the model on top of the linear combination, as the activation function.

For each node within the same hidden layer, are the activation functions usually the same, or differ among each node?

Conventionally, all nodes in the same layer have the same activation. This makes vectorization easy, and ultimately doesn't really affect preformance. Instead of varying activations of individual neurons, complex architectures use parallel layers having different activations but with the same inputs. Usually, all stacked layers in models share the same activations, except maybe the last layer and in some cases, the penultimate layer.