What really means stochastic in field of signal processing I met two definitions of word stochastic, the first one (cited from wikipedia Stochastic)


  The word stochastic is an adjective in English that describes
  something that was randomly determined


The explanation of stochastic and deterministic what is used in textbooks really make sense according to definition above. The second definition is


  Stochastic means sample by sample


For example SGD - stochastic gradient descent.

This seems to be a little confusing for me. I do not really see any relation between random and step by step.

Do I miss something? Or it is really different meanings for the same word?
Well, getting a bit linguistic, according to the Oxford dictionary:


  stochastic (adj.): Having a random probability distribution or pattern that may be analysed statistically but may not be predicted precisely.


So the definition would be the first one (I don't know where you might have found the second one as you didn't put any source about it). Nevertheless, regarding your specific question, SGD is indeed stochastic. It is an iterative method, but this doesn't mean it is not stochastic. It is iterative and stochastic at the same time: the two concepts are not related to each other.

To put it clearer: if something is blue, couldn't it be big? Well... why not? That's exactly what you are asking here. As size isn't related to colour (absurd example that I think can help here), an approximation can be iterative and stochastic (such as SGD): being one thing doesn't mean it can't be the another one too.