How much does the impact factor vary from year to year?
For completeness, I will start by stating the underlying calculation for the impact factor (source): 


total cites in year X
Year X cites to articles published in the two year prior to year X (this is a subset of (1.))
number of articles published in the two years prior to year X
= (2.)/(3.) = impact factor for year X


For most journals the number of papers published per year is relatively constant (item 3.). This means that what in most cases really changes the impact factor is item (2.), how many times papers in the years prior have been cited. This number will vary according to how successful the journal is to attract good papers that can accumulate citations. Under normal circumstances the influx of papers probably follows some form of distribution in terms of "quality" (whatever that is). 

If we accept this as the model (deviations are likely to exist), then a journal with a small number of papers per year will likely be more sensitive to changes in citation rate of a few papers whereas a journal with a large publication rate will likely be less sensitive. 

To change the citation rate of papers in a journal is not easy. The reputation of a journal takes time to build and can be destroyed in a minute. To increase the index is therefore likely more difficult than lowering. Being editor of a small to medium international journal, I have seen that introduction of thematic issues can significantly raise the index. This comes from the fact that such issues constitute a concentration of paper on the same topic which will strongly help each other to attract referenceing; as opposed to a single paper "hidden" in melee of other unrelated papers. So there are means (strategies) to change the index, but for business as usual it will vary but not fluctuate widely. If you find journals where the index varies strongly (more than on the decimal level) it might be interesting to take a closer look and see what factors have influenced the changes.