Is it disrespectful to invite a renowned lecturer if I know that he will be travelling through my city?
This is a completely fair request, and you can ask her/him without any worries. That said, there are some aspects that you may want to take into account to increase the chance of the speaker actually saying yes:


Please do not just show up at the lecture and ask there. Almost certainly (s)he will have made other plans at this point. Ask in advance, and as soon as possible. This also has the advantage that you are not putting her/him on the spot directly, but give time to ponder the issue.
If possible, use a joint acquaintance to put you in contact rather than cold-mailing her/him. The probability of the speaker actually taking time out of her/his day to speak to your faculty will be so much higher this way than if you are just a random stranger who sent her/him a mail.
Frame the meeting as starting point for collaboration, not just an informal chat. Prominent people tend to not be keen on wasting their time on meetings with unclear outcome, so make sure that you and (s)he knows what to expect.
Relatedly, have a clear agenda in mind and make sure in advance that, should the speaker decide to visit you, the key people that you want her/him to meet are actually available. It apparently happens quite frequently that a motivated individual invites a prominent professor over, and then all the other faculty / students are actually not particularly interested in talking to the guest. As you can imagine, this tends to be embarrasing, for both, the guest and the host.