Is stressing too much on formalism acceptable? Most of the students like the analogy way of teaching things. But I am biased towards the formal way along with a little bit of analogy.

For example, (to make simple) in order to teach about a set to the students who don't have any clue about it, I start by saying


  A set is an unordered, well defined collection of elements, in which
  repetition doesn't matter.


Then going with examples of what is a set and what is not. But over time I found that people tend towards analogies, more than formal. I have no issue with analogies and I also do it. But they tend to forget the basic definition and facing problems while doing mathematical analysis.

Colleagues suggest that students can find formal definitions through textbooks and hence reduce burden of teaching too formally by referring corresponding textbooks and focus on motivation and surface teaching only. But I am not at all satisfied by it because in CS, things only work by formal definitions, not (only) by analogy.

What should I do? Continuing or discontinuing on stress formalising?
 (Consider time limitations)
Moving from formal to informal is almost always a mistake.  Let me see if I can show you why.

When you consider how you think about sets within your own mind, I am reasonably certain that you rarely use the formal definition.  Instead, you visualize or otherwise imagine something more akin to some abstract sort of bag, box, or group.  The specifics of your visualizations are not so important here.  However, the internalization that you use must be, in some sense, physical.  Members can be in or out.  Things can be added in, or removed.  Your mental imagery may be only partially fleshed out, or it may be much more definite, with form, color, and texture.

This mental model gives you a strong set of intuitions regarding sets that allow you to work with them quickly, efficiently, and correctly, allowing you to move to formalisms only when you really need them, such as when you are attempting a proof, or when you are trying to enhance your understanding of some novel property that you hadn't understood before.

Why do you suppose that we always draw out our Turing Machines instead of writing out their formal definitions? (I searched online for a long time, and I honestly can't find a single example of a TM entirely written out.)  Similarly, why do we normally draw a geometric diagram instead of simply writing out the formula for a diagonal bisection of a square?

The answer is obviously that the concepts are far eaiser to understand when turned physical.  This property will hold true for basically every abstract concept we've mastered.  Our minds are wired towards the physical.  We can figure out a reasonable path in a crowd of moving people almost without effort, but take months (or even years!) of training to be able to reliably find the sum of multiple-digit numbers.

Taking this idea to an extreme, you could construct a fully formal, mathematical description of Chess.  Squares are sets, numbered from 1-64, and $Pawn_{i}$ where $1 \le i \le 8$ can always move to $Pawn_{i}loc + 8$ iff $current + 8 = \{ \} \land current &lt; 57$. Similar descriptions can be made for every other legal class of moves, but by using such an approach, you would take a game that any 8 year old can learn, and you would make it inscrutable. The poor 8 year old would never be able to learn the mechanics of the game. (And mechanics is not a coincidental word here!)

Visualizations and strong analogies allow us to make concepts instantly clear.  As Buffy points out in his answer, mathematical work moves from concept to formalisms, almost never the other way. To do otherwise is to ignore the way that our minds learn.