How is antibody production stopped?
Upon activation plasma B cells upregulate death receptors as part of being activated. Presence of the antigen overcomes the death signal, thus the cell survives. When antigen is lost, the death signals overcome the survival signals as there is no antigen, so the B cell dies via apoptosis. 

Memory B cells don't do this. They survive and continue to produce antibodies for years, although this slowly wanes. Antibodies have to be produced as antibody mediated immune mechanisms are extremely important for memory and the reason why a subsequent infection is cleared so rapidly. Furthermore for the example of HPV, antibodies induced by the vaccine prevent the virus even infecting cells (so called neutralising antibodies).