Why do people publish on arXiv instead of other places?
The major use case of arXiv is for disseminating manuscripts that you also publish in a journal or conference. By posting a preprint on arXiv, people can find your research, build on it, cite it, and give you feedback on it immediately, while at the same time the same work goes through the (sometimes slow) peer review process. Some of these papers will fall out of the peer review pipeline at some point, and only appear on arXiv, but that doesn't necessarily mean that  they are less useful, important, or sound.

To give a specific example, in the last year or so I have been working in an area so new that most of the relevant research is still only available on arXiv.  I expect these papers will eventually appear in journals too, but the slow peer review process means that the latest journal issues do not represent the state of the art for this particular topic. Conferences have a quicker peer review cycle and are more current, but most only publish relatively short papers.

arXiv is also useful for work that is in a format not suited for a conference or journal (e.g. a thesis), or for extended versions of papers that are published somewhere else.

I would caution against rules like "I should not cite materials on arXiv because they are not peer reviewed." Peer review does not guarantee sound, high quality research, nor is the inverse true.  You should critically evaluate each paper, peer reviewed or not, on its individual merits. (Also see this related question on Math Overflow.)