Can multiple activation functions be replaced with a single activation function?
TL;DR: This is possible but removing the activations will decrease the expressivity of the network because it will become mathematically equivalent to a single neuron.

Mathematical Explanation

The outputs of your intermediate neurons (in the absence of activation functions) are now:

$\text{(1)}\quad H_i(x) = \sum_{j=1}^nw_{ij}\cdot x_j+b_i$

You are then summing each $H_i$:

$\text{(2)}\quad\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$

You then pass $\mathcal{H(x)}$ to some activation say $g$:

$\text{(3)}\quad\hat y = g(\mathcal{H(x)})$

The trouble is that the inner term $\mathcal{H(x)}$ mathematically reduces to a single linear operation on $x$. Proof:


$\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$. Substituting in (1):
$\mathcal{H(x)}=\sum_{i=1}^n((\sum_{j=1}^nw_{ij}\cdot x_j)+b_i)$. This can be re-aranged :
$\mathcal{H(x)}=(\sum_{j=1}^n(\sum_{i=1}^nw_{ij})\cdot x_j)+\sum_{i=1}^nb_i$. But this reduces to:
$\mathcal{H(x)}=\sum_{j=1}^n\tilde w_{j}\cdot x_j+\tilde b$. Where $\tilde w_{j},\tilde b$ are scalars. 


Thus, without the non-linear activations (3) mathematically reduces to a single neuron.