What is the difference between a bias and a heuristic explained in layman terms? Can anyone explain the difference in a way which can be explained to some one who does not a have a clue about psychology? May be examples could help.
A heuristic is an approach to problem solving, a bias is a prejudice; so in what way do these terms confuse you?


I respectfully disagree. I have noticed that the term bias and heuristic are used interchangeably in the literature that could lead to confusion. The difference between them is subtle.


  Can anyone explain the difference in a way which can be explained to some one who does not a have 


Heuristics, biases and algorithms are all related terms. The simplest way to describe them is as follows:

A heuristic is a rule, strategy or similar mental shortcut that one can use to derive a solution to a problem. A heuristic that works all of the time is known as an algorithm. Consider the following scenario: you get lost in a maze, what can you do to escape? The solution is to place your hand on the outer edge of the maze and keep walking until you find the exit. Heuristics can help us make sense of the world in a reliable way and reduce our mental load. However, they aren't always perfect and there is also a tendency to use them inappropriately to form beliefs. A systematic error that results from the use of a heuristic is called a cognitive bias. 

EDIT:

Per suggestions, the following should be noted:

While the use of heuristics of can lead to cognitive biases, not all cognitive biases  are the result of heuristics. Generally, however, when an error in judgement or belief is identified empirically or anecdotally, there will be an attempt to account for it using heuristics.

Additionally, and more broadly, there is just as much ambiguity regarding the definition of cognitive bias in the literature. 

For example, in this account of attribution biases, the following definition is provided:


  Cognitive biases refer to systematic mistakes that derive from limits
  that are inherent in our capacity to process information. Because we
  are not capable of perceiving everything in our environment, our focus
  is automatically drawn to the most prominent or “eye-catching”—that
  is, perceptually salient—stimuli. This can lead us to formulate biased
  and inaccurate causal attributions (Taylor &amp; Fiske, 1975).
  Specifically, we are prone to equate the most perceptually salient
  stimuli with the most causally influential stimuli.  
  
  Shiraev, E. B., Shiraev, E. B., &amp; Levy, D. A. (2016). Cross-cultural psychology: Critical thinking and contemporary applications. Taylor &amp; Francis.


Similarly:


  When the correct answer is described, they see for themselves how
  their own judgment is biased in systematic ways. Such personal
  demonstrations make Tversky and Kahneman's points in a salient,
  simple, and powerful way. What was innovative about their work was
  that it drew a connection between heuristics and biases: A heuristic
  is a rule of thumb used as such in different contexts. A cognitive
  bias is a systematic error in our thinking.
  
  Morvan, C., &amp; Jenkins, W. J. (2017). Judgment Under Uncertainty: Heuristics and Biases. CRC Press.


However:


  Heuristics are the "shortcuts" that humans use to reduce task
  complexity in judgment and choice, and biases are the resulting gaps
  between normative behavior and the heuristically determined behavior
  (Kahneman et al., 1982).
  
  Chipman, S. E. (Ed.). (2016). The Oxford Handbook of Cognitive Science. Oxford University Press.


Ditto:


  Cognitive biases are subconscious deviations in judgement leading to
  perceptual distortion, inaccurate judgement and illogical
  interpretation. From an evolutionary point of view, they have
  developed because often speed was more important than accuracy. Biases
  occur due to information processing shortcuts (or heuristics – see Box
  5.1), the brain's limited capacity to process information, social 
  influence, and emotional and moral motivations.
  
  Cooper, N., &amp; Frain, J. (Eds.). (2016). ABC of clinical reasoning. John Wiley &amp; Sons.


Although this isn't a psychology reference, the definition is a tad weird: 


  Architects rely on precedent knowledge to design their next
  building, believing the past buildings worked well and the knowledge
  can be transferred to their new design, promising similar
  performances. Cognitive biases (illusions) and potential errors can
  occur when using precedent knowledge for analogical, pre-parametric
  and qualitative design thinking.
  
  Zarzar, K. M., &amp; Guney, A. (Eds.). (2008). Understanding Meaningful Environments: Architectural Precedents and the Question of Identity in Creative Design (Vol. 4). IOS Press.


Although I am speculating, I would suggest the following explanation: strictly speaking, an error in normative reasoning need only occur once for it to potentially be the result of a cognitive bias. However, a cognitive bias would need to occur in a systematic manner in a study sample before it could be identified in empirical research and reported in the literature.

And that, my friends, concludes my demonstration on overkill.