How to Prevent continuous switching fluctuation?
A simple approach is to say that if your input is above a higher threshold, you should switch the relay one way, if it's below a lower threshold you should switch it the other, and if it's between the thresholds you should leave the relay alone.  One limitation of this approach from a control-system perspective is that if the system is sometimes noisy it may be necessary to set the thresholds relatively far apart in order to avoid "chatter", and setting the thresholds far apart may make the system response "sloppy" when things are quiet.

A slightly better approach is to use a counter variable to keep track of when it would be worthwhile to switch the relay.  Instead of using the two thresholds to switch the relay directly, say that when the input is beyond the threshold which would indicate that the relay should be on, increment a counter if it's below 255.  When it's beyond the threshold which would indicate that the relay should stay where it is, decrement the counter if not zero.  If it's between the thresholds, leave the counter where it is.  If the counter reaches zero and the relay isn't already off, turn it off.  Likewise if the counter reaches 255 and the relay isn't already on.

Using this approach, if the input is above the "turn relay on" threshold more often than it's below the "turn relay off" threshold, the relay will eventually turn on.  Likewise if it's below the "turn relay off" threshold more often than above "turn relay on".  The distance between the minimum and maximum counter values may be adjusted as needed to control how often the relay will switch.  Because the system will only switch the relay if it sees a persistent balance of input values indicating "turn on" or "turn off", the thresholds can be set much closer together than in the former scenario.