Adam Optimiser First Step
These are the equation of Adam [Ref - Dive Into Deep Learning]
\begin{aligned}
       \mathbf{v}_t &amp; \leftarrow \beta_1 \mathbf{v}_{t-1} + (1 - \beta_1) \mathbf{g}_t \\
       \mathbf{s}_t &amp; \leftarrow \beta_2 \mathbf{s}_{t-1} + (1 - \beta_2) \mathbf{g}_t^2
\end{aligned}
\begin{aligned}
\hat{\mathbf{v}}_t = \frac{\mathbf{v}_t}{1 - \beta_1^t} \text{ and } \hat{\mathbf{s}}_t = \frac{\mathbf{s}_t}{1 - \beta_2^t}
\end{aligned}
\begin{aligned}
\mathbf{g}_t' = \frac{\eta \hat{\mathbf{v}}_t}{\sqrt{\hat{\mathbf{s}}_t} + \epsilon}
\end{aligned}
\begin{aligned}
\mathbf{x}_t \leftarrow \mathbf{x}_{t-1} - \mathbf{g}_t'
\end{aligned}

The first two are the accumulation of momentum and the second moment of the gradient
The second set is for correction of initial bias
Last two are parameter update

Initial values are - [Ref - Arxiv Paper]
\begin{aligned}
       \mathbf{v} = \mathbf{s} = 0; \mathbf{t} = 1^{**} ; \beta_1=0.9 ; \beta_2=0.999 ;  \epsilon = 10^{-8}
\end{aligned}
Note - ** - It is initialized with 0 but incremeted in the loop before any other operation
These defaults will make,
\begin{aligned}
\mathbf{g}_0' = \eta \text{  (approximated for  } \epsilon\text{ )}
\end{aligned}
So, the initial movement will not be proportional to the Gradient.