Should I include "visiting" in my visiting assistant professor, or just call myself assistant professor?
I agree with you.  People expect websites, email signatures, etc, to have accurate job titles, and there is a major qualitative difference between "Visiting Assistant Professor" and "Assistant Professor".  Dropping the "Visiting" unintentionally is careless, and intentionally dropping it to "upsell" (i.e. mislead) is unethical.

(You might as well say, as long as you're dropping words, why not just make yourself a full "Professor"?)